-- Logs begin at Sun 2022-10-30 00:10:42 UTC, end at Sun 2022-10-30 01:21:40 UTC. -- Oct 30 00:47:11.762687 dv-cinder-master-3-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Oct 30 00:47:11.771515 dv-cinder-master-3-iscsi devstack@n-api.service[103758]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Oct 30 00:47:11.773445 dv-cinder-master-3-iscsi devstack@n-api.service[103758]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sun Oct 30 00:47:11 2022] *** Oct 30 00:47:11.773445 dv-cinder-master-3-iscsi devstack@n-api.service[103758]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 30 00:47:11.773445 dv-cinder-master-3-iscsi devstack@n-api.service[103758]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 30 00:47:11.773445 dv-cinder-master-3-iscsi devstack@n-api.service[103758]: nodename: dv-cinder-master-3-iscsi Oct 30 00:47:11.773445 dv-cinder-master-3-iscsi devstack@n-api.service[103758]: machine: x86_64 Oct 30 00:47:11.773445 dv-cinder-master-3-iscsi devstack@n-api.service[103758]: clock source: unix Oct 30 00:47:11.773445 dv-cinder-master-3-iscsi devstack@n-api.service[103758]: pcre jit disabled Oct 30 00:47:11.773445 dv-cinder-master-3-iscsi devstack@n-api.service[103758]: detected number of CPU cores: 4 Oct 30 00:47:11.773445 dv-cinder-master-3-iscsi devstack@n-api.service[103758]: current working directory: / Oct 30 00:47:11.773445 dv-cinder-master-3-iscsi devstack@n-api.service[103758]: detected binary path: /usr/bin/uwsgi-core Oct 30 00:47:11.773445 dv-cinder-master-3-iscsi devstack@n-api.service[103758]: your processes number limit is 47680 Oct 30 00:47:11.773445 dv-cinder-master-3-iscsi devstack@n-api.service[103758]: your memory page size is 4096 bytes Oct 30 00:47:11.773445 dv-cinder-master-3-iscsi devstack@n-api.service[103758]: detected max file descriptor number: 2048 Oct 30 00:47:11.773445 dv-cinder-master-3-iscsi devstack@n-api.service[103758]: lock engine: pthread robust mutexes Oct 30 00:47:11.774372 dv-cinder-master-3-iscsi devstack@n-api.service[103758]: thunder lock: enabled Oct 30 00:47:11.774372 dv-cinder-master-3-iscsi devstack@n-api.service[103758]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Oct 30 00:47:11.774372 dv-cinder-master-3-iscsi devstack@n-api.service[103758]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 30 00:47:11.812162 dv-cinder-master-3-iscsi devstack@n-api.service[103758]: Python main interpreter initialized at 0x558d3b9839c0 Oct 30 00:47:11.812162 dv-cinder-master-3-iscsi devstack@n-api.service[103758]: python threads support enabled Oct 30 00:47:11.812162 dv-cinder-master-3-iscsi devstack@n-api.service[103758]: your server socket listen backlog is limited to 100 connections Oct 30 00:47:11.812162 dv-cinder-master-3-iscsi devstack@n-api.service[103758]: your mercy for graceful operations on workers is 90 seconds Oct 30 00:47:11.812509 dv-cinder-master-3-iscsi devstack@n-api.service[103758]: mapped 403077 bytes (393 KB) for 2 cores Oct 30 00:47:11.812509 dv-cinder-master-3-iscsi devstack@n-api.service[103758]: *** Operational MODE: preforking *** Oct 30 00:47:11.812809 dv-cinder-master-3-iscsi devstack@n-api.service[103758]: *** uWSGI is running in multiple interpreter mode *** Oct 30 00:47:11.812809 dv-cinder-master-3-iscsi devstack@n-api.service[103758]: spawned uWSGI master process (pid: 103758) Oct 30 00:47:11.813003 dv-cinder-master-3-iscsi systemd[1]: Started Devstack devstack@n-api.service. Oct 30 00:47:11.814004 dv-cinder-master-3-iscsi devstack@n-api.service[103758]: spawned uWSGI worker 1 (pid: 103759, cores: 1) Oct 30 00:47:11.814004 dv-cinder-master-3-iscsi devstack@n-api.service[103758]: spawned uWSGI worker 2 (pid: 103760, cores: 1) Oct 30 00:47:11.814004 dv-cinder-master-3-iscsi devstack@n-api.service[103758]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 30 00:47:12.458666 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Oct 30 00:47:12.461019 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Oct 30 00:47:13.517546 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 30 00:47:13.517546 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: from cryptography.utils import int_from_bytes Oct 30 00:47:13.518252 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 30 00:47:13.518252 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: from cryptography.utils import int_from_bytes Oct 30 00:47:13.518252 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 30 00:47:13.518252 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: from cryptography.utils import int_from_bytes Oct 30 00:47:13.518718 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 30 00:47:13.518718 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: from cryptography.utils import int_from_bytes Oct 30 00:47:15.977352 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 30 00:47:15.977950 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 30 00:47:15.978094 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 30 00:47:15.978477 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 30 00:47:15.978609 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 30 00:47:15.978995 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 30 00:47:15.979084 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 30 00:47:15.979175 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.979336 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 30 00:47:15.979424 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.979696 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.979854 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 30 00:47:15.980006 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 30 00:47:15.980147 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.980346 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.980498 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.980694 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.980793 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.980949 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.981057 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.981277 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.981422 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.981595 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.981694 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.981841 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.981937 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.982157 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.982257 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.982482 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.982571 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.982790 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.982876 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.983059 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.983147 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.983368 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.983466 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.983676 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.983786 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.984002 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.984090 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.984315 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.984414 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.984613 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.984712 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.984925 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.985085 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.985212 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.986274 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.986415 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.986571 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.986659 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.986738 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.986822 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.986918 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.987006 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.987094 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.987172 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.987296 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.987383 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.987470 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.987564 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.987655 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.987850 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.987941 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.988158 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.988260 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.988473 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.988612 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.988767 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.988904 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 4.0 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.989086 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.989211 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.989395 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.989516 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.0 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.989800 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.989945 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.990104 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 4.0 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.990245 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.990414 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.990519 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.990724 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.0 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.990811 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.991107 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.991483 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.991622 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.991705 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.991861 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.991951 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.992139 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.992233 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.992468 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.992567 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.992775 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.992860 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.993103 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.993194 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.993408 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.993493 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.993715 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.993801 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.994033 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.994124 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.994334 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.994417 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.994632 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.994721 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.994878 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.994962 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.995182 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.995267 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.995499 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.995583 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.995839 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.995940 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.996088 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.996174 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.996394 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.996487 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.996638 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.996725 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.996947 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.997031 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.997259 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.997357 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.997563 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.997653 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.997810 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.997907 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.998125 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.998210 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.998441 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.998532 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.998741 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.998837 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.999055 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.999179 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.999338 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.999431 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.999640 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.999746 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:15.999973 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.000071 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.000276 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.000361 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.000580 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.000672 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.000881 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.000965 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.001193 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.001296 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.001505 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.001603 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.16 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.001810 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.16 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.001903 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.002112 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.002204 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.002426 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.002517 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.002739 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.002823 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.16 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.003044 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.003128 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.16 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.003350 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.003433 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.003660 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.003795 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.003990 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.004078 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.004292 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.004377 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.004596 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.004679 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.004910 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.005000 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.005216 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.005301 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.005520 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.005604 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.005762 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.005855 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.006059 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.006195 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.006363 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.006449 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.006675 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.006767 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.007010 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.007094 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.007311 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.007401 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.007610 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.007704 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.008008 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.008102 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.008314 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.008398 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.008618 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.008780 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.008918 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.009000 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.009217 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.009299 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.009517 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.009610 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.009819 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.009911 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.010107 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.010203 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.010404 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.010497 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.010706 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.010789 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.011003 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.011087 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.011304 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.011394 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.011612 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.011827 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.011959 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.012090 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.012235 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.012366 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.012510 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.012642 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.012799 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.012930 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.013113 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.013254 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.013414 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.013550 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.013702 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.013834 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.014005 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.014135 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.014301 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.014438 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.014604 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.014743 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.014908 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.015038 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.015206 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.015337 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.015505 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.015656 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.015819 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.015952 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.016129 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.016275 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.016432 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.016558 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.016739 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.016867 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.017039 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.017176 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.017339 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.017470 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.017618 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.017754 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.017918 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.018056 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.018214 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.018354 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.018511 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.018642 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.018808 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.018935 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.019105 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.019248 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.019392 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.019527 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.019693 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.019853 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.020015 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.020172 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.020298 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.020426 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.020600 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.020724 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.020897 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.021030 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.021210 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.021359 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.021513 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.021651 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.021820 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.021952 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.022123 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.022249 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.022481 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:47:16.022617 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.022862 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.022991 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.023190 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.023325 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.023505 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.023637 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.023837 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.023968 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.024150 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.024277 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.024472 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.024607 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.024778 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.024910 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.025083 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.025210 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.025388 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.025523 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.025702 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.025828 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.026014 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.026152 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.026362 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.026521 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.026656 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.026784 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.026938 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.027065 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.027250 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.027395 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.027558 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.027685 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.027896 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.028023 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.028215 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.028355 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.028526 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.028662 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.028831 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.028963 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.029144 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.029364 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.029497 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.029624 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.029777 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.029909 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.030088 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.030219 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.030400 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.030487 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.030708 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.030795 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.031014 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.031100 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.031319 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.031403 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.031653 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.031760 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.031989 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.032075 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.032299 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.032381 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.032610 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.032703 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.032917 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.033002 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.033235 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.033319 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.033544 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.033636 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.033851 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.033943 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.034157 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.034243 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.034464 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.034592 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.034767 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.034850 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.035075 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.035163 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.035385 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.035483 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.035696 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.035813 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.036030 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.036121 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.036344 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.036479 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.036656 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.036741 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.036964 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.037047 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.037271 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.037362 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.037575 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.037659 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.037873 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.037963 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.038213 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.038297 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.038515 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.038622 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.038819 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.038939 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.039118 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.039247 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.039433 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.039554 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.039772 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.039910 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.040075 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.040203 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.040403 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.040531 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.040704 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.040845 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.041008 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.041135 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.041314 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.041446 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.041620 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.041757 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.041920 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.042048 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.042225 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.042360 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.042531 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.042666 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.042832 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.042960 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.043146 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.043282 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.043449 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.043589 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.043751 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.043891 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.044070 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.044208 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.044368 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.044508 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.044671 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.044807 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.044976 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.045121 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.045265 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.045397 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.045569 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.045695 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.045876 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.046010 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.046181 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.046312 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.046490 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.046622 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.046797 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.046924 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.047101 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.047230 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.047405 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.047531 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.047763 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.047895 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.048040 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.048168 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.048353 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.048490 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.048673 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.048812 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.048984 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.049132 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.049282 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.049414 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.049587 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.049727 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.049888 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.050022 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.050187 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.050315 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.050484 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.050622 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.050786 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.050914 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.051084 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.051211 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.051382 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.051523 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.051679 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.051829 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.052000 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.052136 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.052282 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.052420 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.052588 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.052718 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.052888 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.053022 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.053185 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.053311 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.053481 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.053613 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.053864 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.054003 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.054185 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.054369 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.054488 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.054676 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.054796 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.055019 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.055117 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.055344 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.055436 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.055658 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.055762 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.056002 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.056097 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.056312 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.056396 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.056629 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.056719 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.056935 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.057020 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.057240 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.057323 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.057544 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.057734 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.057894 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.057975 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.058153 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.058237 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.058456 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.058539 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.058765 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.058857 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.059069 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.059151 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.059381 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.059464 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.059684 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.059850 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.060018 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.060101 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.060316 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.060417 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.060625 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.060720 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.060937 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.061021 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.061240 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.061331 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.061553 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.061651 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.061868 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.061952 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.062172 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.062264 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.062480 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.062567 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.062783 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.062867 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.063087 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.063179 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.063390 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.063475 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.063692 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.063797 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.064023 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.064109 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.064327 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.064422 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.064630 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.064715 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.064931 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.065017 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.065236 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.065318 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.065537 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.065619 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.065848 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.065931 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.066144 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.066239 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.066457 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.066541 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.066774 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.066865 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.067428 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.067520 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.067757 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.067880 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.068065 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.068168 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.068378 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.068475 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.068681 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.068787 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.068985 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.069081 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.069289 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.069397 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.069617 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.069762 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.069899 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.069969 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.070206 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.070303 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.070516 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.070631 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.070815 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.070929 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.071114 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.071207 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.071411 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.071507 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.071729 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.071841 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.072037 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.072138 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.072334 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.072424 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.072632 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.072719 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.072935 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.073020 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.073236 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.073331 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.073535 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.073621 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.073842 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.073928 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.074153 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.074241 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.074452 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.074538 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.074750 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.074836 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.075069 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.075168 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.075366 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.075453 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.075661 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.075853 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.075985 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.076073 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.076282 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.076370 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.076589 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.076696 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.076893 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.076994 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.077192 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.077281 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.077499 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.077600 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.077813 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.077900 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.078115 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.078202 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.078419 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.078531 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.078722 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.078810 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.079024 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.079121 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.079333 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.079420 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.079638 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.079743 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.079962 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.080049 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.080271 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.080360 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.080575 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.080661 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.080877 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.080964 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.081181 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.081269 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.081486 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.081576 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.081786 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.081872 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.082089 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.082177 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.082393 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.082478 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.082689 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.082783 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.082992 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.083087 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.083303 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.083389 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.083603 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.083886 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.083979 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.084215 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.084302 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.084525 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.084802 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.084897 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.085198 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.085341 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.085601 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.085766 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.085882 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.085970 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.086206 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.086294 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.086521 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.086656 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.086822 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.086925 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.087129 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.087238 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.087425 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.087523 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.087735 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.087826 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.088049 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.088155 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.088348 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.088435 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.088645 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.088731 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.088943 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.089039 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.089243 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.089330 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.089538 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.089626 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.089836 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.089923 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.090135 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.090222 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.090435 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.090521 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.090737 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.090825 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.091033 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.091122 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.091334 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.091469 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.091619 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.091772 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.091958 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.092093 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.092261 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.092395 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.092561 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.092700 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.092859 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.093014 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.093173 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.093312 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.093446 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.093584 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.093745 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.093887 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.094048 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.094191 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.094387 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.094529 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.094690 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.094826 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.095005 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.095147 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.095294 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.095424 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.095606 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.095783 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.095972 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.096117 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.096274 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.096409 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.096575 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.096709 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.096881 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.097017 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.097176 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.097304 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.097468 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.097603 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.097772 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.097920 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.098075 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.098170 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.098374 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.098461 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.098674 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.098761 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.098971 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.099059 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.099268 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.099417 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.099566 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.099657 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.099888 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.099976 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.100188 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.100292 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.100487 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.100573 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.100843 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.100932 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.101148 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.101317 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.101449 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.101624 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.101746 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.101929 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.102048 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.102238 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.102333 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.102544 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.102632 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.102845 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.102932 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.103153 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.103241 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.103454 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.103541 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.103791 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.103897 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.104093 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.104187 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.104389 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.104476 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.104689 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.104784 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.104995 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.105081 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.105293 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.105382 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.105591 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.105686 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.105895 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.105984 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.106193 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.106281 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.106495 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.106591 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.106794 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.106880 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.107101 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.107197 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.107404 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.107500 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.107696 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.107847 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.108018 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.108114 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.108316 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.108404 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.108614 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.108701 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.108910 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.109005 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.109212 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.109307 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.109508 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.109594 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.109804 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.109892 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.110100 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.110232 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.110396 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.110483 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.110691 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.110791 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.110989 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.111074 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.111287 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.111373 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.111582 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.111685 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.111902 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.111992 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.112200 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.112288 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.112511 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.112605 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.112813 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.112900 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.113114 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.113200 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.113416 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.113516 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.113739 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.113825 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.114038 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.114123 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.114341 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.114428 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.114645 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.114743 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.114953 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.115040 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.115250 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.115339 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.115559 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.115649 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.115886 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.115977 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.116219 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.116306 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.116568 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.116701 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.116883 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.117041 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.117195 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.117361 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.117511 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.117672 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.117818 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.117980 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.118124 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.118298 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.118444 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.118616 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.118757 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.118940 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.119066 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.119240 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.119386 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.119545 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.119706 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.119878 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.120053 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.120183 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.120358 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.120489 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.120669 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.120796 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.120977 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.121103 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.121288 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.121415 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.121598 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.121751 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.121910 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.122041 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.122223 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.122359 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.122536 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.122673 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.122846 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.122980 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.123155 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.123299 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.123450 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.123582 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.123765 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.123855 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.124069 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.124169 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.124370 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.124458 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.124673 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.124759 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.124971 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.125056 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.125278 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.125366 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.125578 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.125664 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.125881 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.125995 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.126187 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.126305 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.126503 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.126610 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.126811 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.126906 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.127199 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.127285 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.127503 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.127593 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.127839 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.127943 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.128143 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.128229 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.128442 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.128529 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.128749 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.128845 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.129061 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.129148 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.129372 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.129458 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.129678 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.129765 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.130013 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.130106 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.130311 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.130397 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.130616 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.130704 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.130933 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.131019 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.131230 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.131353 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.131533 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.131620 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.131896 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.131992 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.132220 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.132340 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.132523 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.132614 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.132827 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.132913 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.133135 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.133272 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.133453 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.133579 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.133756 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.133879 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.134075 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.134285 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.134486 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.134592 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.134795 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.134901 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.135099 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.135218 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.135508 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.135694 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.135840 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.135967 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.136149 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.136248 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.136474 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.136571 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.136791 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.136879 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.137110 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.137197 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.137412 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.137516 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.137721 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.137857 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.138026 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.138161 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.138323 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.138471 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.138640 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.138777 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.138950 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.139083 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.139257 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.139391 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.139560 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.139694 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.139909 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.140046 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.140231 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.140367 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.140544 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.140688 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.140865 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.141001 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.141192 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.141335 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.141482 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.141623 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.141784 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.141936 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.142090 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.142234 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.142393 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.142544 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.142678 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.142818 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.142995 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.143131 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.143297 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.143438 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.143604 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.143779 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.143937 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.144063 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.144225 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.144361 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.144517 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.144659 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.144813 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.144949 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.145128 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.145261 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.145403 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.145536 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.145717 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.145849 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.146024 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.146160 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.146337 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.146477 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.146639 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.146781 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.146936 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.147069 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.147250 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.147405 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.147643 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.147857 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.147989 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.148120 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.148292 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.148447 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.148615 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.148751 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.148924 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.149149 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.149284 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.149421 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.149554 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.149695 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.149828 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.149972 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.150129 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.150264 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.150437 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.150569 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.150755 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.150900 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.151082 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.151219 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.151407 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.151499 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.151708 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.151817 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.152056 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.152147 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.152374 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.152462 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.152674 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.152762 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.152971 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.153059 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.153275 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.153362 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.153572 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.153661 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.153869 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.153957 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.154167 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.154256 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.154469 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.154557 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.154768 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.154855 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.155066 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.155162 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.155360 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.155447 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.155662 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.155800 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.155982 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.156083 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.156282 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.156558 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.156683 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.156799 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.156921 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.157019 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.157160 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.157248 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.157455 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.157543 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.157779 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.157916 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.158075 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.158162 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.158375 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.158461 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.158677 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.158772 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.158989 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.159076 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.159301 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.159396 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.159611 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.159699 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.159942 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.160028 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.160248 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.160344 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.160555 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.160652 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.160864 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.160951 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.161172 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.161266 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.161478 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.161566 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.161788 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.161879 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.162103 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.162190 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.162411 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.162506 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.162716 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.162928 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.163015 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.163295 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.163392 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.163671 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.163777 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.164014 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.164215 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.164318 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.164544 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.164639 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.164855 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.164952 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.165170 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.165267 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.165482 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.165577 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.165796 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.165879 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.166103 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.166192 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.166417 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.166512 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.166729 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.166818 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.167037 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.167127 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.167347 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.167434 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.167662 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.167775 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.168006 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.168095 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.168332 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.168420 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.168644 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.168739 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.168972 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.169067 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.169285 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.169373 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.169587 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.169683 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.169894 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.169990 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.170204 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.170292 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.170513 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.170601 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.170819 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.170913 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.171124 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.171212 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.171435 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.171521 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.171755 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.171857 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.172063 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.172198 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.172370 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.172457 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.172674 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.172761 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.172981 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.173081 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.173287 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.173374 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.173595 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.173683 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.173905 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.174005 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.174211 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.174297 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.174519 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.174606 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.174839 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.174935 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.175156 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.175343 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.175470 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.175653 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.175791 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.176006 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.176102 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.176325 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.176623 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.176710 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.176935 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.177025 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.177247 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.177334 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.177565 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.177652 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.177868 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.177958 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.178170 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.178262 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.178475 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.178581 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.178858 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.178969 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.179167 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.179255 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.179472 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.179569 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.179802 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.179905 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.180114 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.180203 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.180417 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.180517 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.180735 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.180823 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.181043 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.181274 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.181361 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.181617 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.181706 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.181933 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.182029 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.182252 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.182462 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.182558 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.182811 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.182900 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.183131 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.183218 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.183456 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.183544 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.183781 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.183872 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.184084 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.184174 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.184394 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.184482 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.184702 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.184790 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.184999 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.185086 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.185297 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.185386 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.185601 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.185690 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.185901 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.185997 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.186201 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.186304 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.186503 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.186591 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.186800 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.186896 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.187106 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.187201 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.187404 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.187491 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.187702 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.187831 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.188020 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.188108 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.188326 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.188413 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.188625 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.188722 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.188923 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.189023 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.189223 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.189319 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.189521 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.189616 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.189827 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.189914 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.190124 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.190210 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.190425 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.190519 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.190720 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.190853 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.191025 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.191114 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.191324 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.191415 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.191629 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.191747 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.191956 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.192404 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.192593 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.192682 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.192895 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.192982 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.193195 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.193291 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.193505 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.193593 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.193811 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.193899 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.194128 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.194233 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.194488 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.194579 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.194804 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.194895 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.195112 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.195208 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.195433 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.195532 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.195757 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.196001 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.196100 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.196333 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.196431 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.196647 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.196743 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.196973 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.197217 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.197305 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.197546 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.197632 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.197868 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.197955 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.198165 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.198298 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.198484 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.198572 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.198787 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.198876 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.199090 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.199176 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.199393 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.199529 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.199695 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.199826 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.200023 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.200112 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.200326 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.200413 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.200631 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.200726 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.200930 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.201016 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.201237 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.201324 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.201537 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.201633 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.201839 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.201934 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.202143 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.202238 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.202443 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.202540 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.202743 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.202830 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.203048 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.203135 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.203360 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.203449 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.203656 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.203764 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.203997 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.204103 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.204307 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.204395 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.204614 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.204709 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.204917 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.205014 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.205220 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.205308 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.205531 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.205620 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.205836 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.205935 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.206142 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.206228 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.206456 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.206547 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.206761 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.206935 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.207060 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.207156 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.207382 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.207698 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.207828 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.208031 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.208119 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.208331 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.208417 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.208652 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.208957 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.209044 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.209274 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.209361 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.209591 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.209677 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.209893 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.209998 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.210247 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.210333 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.210569 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.210656 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.210889 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.210978 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.211198 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.211285 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.211515 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.211703 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.211848 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.211937 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.212155 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.212242 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.212459 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.212559 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.212759 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.212848 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.213062 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.213150 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.213376 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.213476 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.213681 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.213769 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.213980 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.214074 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.214278 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.214379 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.214580 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.214674 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.214880 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.214967 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.215178 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.215266 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.215477 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.215607 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.215806 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.215897 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.216110 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.216199 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.216404 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.216490 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.216703 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.216798 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.217000 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.217090 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.217296 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.217383 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.217593 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.217680 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.217890 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.217978 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.218187 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.218273 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.218483 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.218577 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.218779 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.218873 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.219074 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.219160 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.219370 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.219472 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.219666 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.219786 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.219992 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.220079 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.220292 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.220392 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.220591 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.220679 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.220890 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.220978 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.221212 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.221299 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.221527 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.221614 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.221829 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.221926 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.222144 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.222274 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.222454 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.222548 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.222761 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.222886 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.223063 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.223154 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.223380 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.223476 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.223680 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.223799 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.224006 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.224094 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.224308 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.224395 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.224611 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.224705 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.224912 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.224998 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.225214 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.225300 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.225521 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.225677 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.225881 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.225976 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.226189 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.226274 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.226504 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.226663 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.226824 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.226961 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.227146 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.227283 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.227496 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.227649 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.227844 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.227933 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.228081 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.228183 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.228387 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.228483 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.228689 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.228783 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.228987 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.229075 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.229291 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.229388 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.229596 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.229684 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.229902 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.229998 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.230203 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.230291 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.230508 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.230595 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.230812 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.230906 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.231121 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.231208 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.231425 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.231511 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.231747 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.231853 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.232059 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.232145 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.232370 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.232456 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.232687 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.232819 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.233008 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.233095 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.233315 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.233402 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.233637 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.233725 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.233959 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.234046 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.234268 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.234357 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.234573 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.234663 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.234873 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.234961 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.235110 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103760) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 30 00:47:16.235237 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.235550 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.235891 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.236200 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:47:16.236442 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 30 00:47:16.391203 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: INFO dbcounter [None req-a4281e05-8cd0-456c-bec0-7b735525da63 None None] Registered counter for database nova_api Oct 30 00:47:16.392650 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: INFO dbcounter [None req-1065301a-c3cb-44d0-8722-d91c9712f6a9 None None] Registered counter for database nova_api Oct 30 00:47:16.415510 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG oslo_db.sqlalchemy.engines [None req-a4281e05-8cd0-456c-bec0-7b735525da63 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103760) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 30 00:47:16.417532 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG oslo_db.sqlalchemy.engines [None req-1065301a-c3cb-44d0-8722-d91c9712f6a9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103759) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 30 00:47:16.423448 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG dbcounter [-] [103760] Writer thread running {{(pid=103760) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 30 00:47:16.424202 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG dbcounter [-] [103759] Writer thread running {{(pid=103759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 30 00:47:16.480198 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.context [None req-a4281e05-8cd0-456c-bec0-7b735525da63 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),a2b915ba-4703-4df7-87fc-723ef53ef868(cell1) {{(pid=103760) load_cells /opt/stack/nova/nova/context.py:464}} Oct 30 00:47:16.480639 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.context [None req-1065301a-c3cb-44d0-8722-d91c9712f6a9 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),a2b915ba-4703-4df7-87fc-723ef53ef868(cell1) {{(pid=103759) load_cells /opt/stack/nova/nova/context.py:464}} Oct 30 00:47:16.484018 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG oslo_concurrency.lockutils [None req-a4281e05-8cd0-456c-bec0-7b735525da63 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103760) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:47:16.484489 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG oslo_concurrency.lockutils [None req-a4281e05-8cd0-456c-bec0-7b735525da63 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=103760) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:47:16.484646 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG oslo_concurrency.lockutils [None req-1065301a-c3cb-44d0-8722-d91c9712f6a9 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103759) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:47:16.484994 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG oslo_concurrency.lockutils [None req-1065301a-c3cb-44d0-8722-d91c9712f6a9 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103759) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:47:16.486104 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG oslo_concurrency.lockutils [None req-a4281e05-8cd0-456c-bec0-7b735525da63 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103760) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:47:16.486636 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG oslo_concurrency.lockutils [None req-1065301a-c3cb-44d0-8722-d91c9712f6a9 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103759) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:47:16.486785 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG oslo_concurrency.lockutils [None req-a4281e05-8cd0-456c-bec0-7b735525da63 None None] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103760) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:47:16.487240 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG oslo_concurrency.lockutils [None req-a4281e05-8cd0-456c-bec0-7b735525da63 None None] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103760) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:47:16.487395 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG oslo_concurrency.lockutils [None req-1065301a-c3cb-44d0-8722-d91c9712f6a9 None None] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103759) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:47:16.487772 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG oslo_concurrency.lockutils [None req-1065301a-c3cb-44d0-8722-d91c9712f6a9 None None] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103759) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:47:16.489271 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG oslo_concurrency.lockutils [None req-a4281e05-8cd0-456c-bec0-7b735525da63 None None] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103760) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:47:16.489796 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG oslo_concurrency.lockutils [None req-1065301a-c3cb-44d0-8722-d91c9712f6a9 None None] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103759) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:47:16.491378 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: INFO dbcounter [None req-a4281e05-8cd0-456c-bec0-7b735525da63 None None] Registered counter for database nova_cell0 Oct 30 00:47:16.491882 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: INFO dbcounter [None req-1065301a-c3cb-44d0-8722-d91c9712f6a9 None None] Registered counter for database nova_cell0 Oct 30 00:47:16.495482 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: INFO dbcounter [None req-a4281e05-8cd0-456c-bec0-7b735525da63 None None] Registered counter for database nova_cell1 Oct 30 00:47:16.496046 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: INFO dbcounter [None req-1065301a-c3cb-44d0-8722-d91c9712f6a9 None None] Registered counter for database nova_cell1 Oct 30 00:47:16.503328 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG oslo_db.sqlalchemy.engines [None req-a4281e05-8cd0-456c-bec0-7b735525da63 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103760) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 30 00:47:16.505134 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG oslo_db.sqlalchemy.engines [None req-a4281e05-8cd0-456c-bec0-7b735525da63 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103760) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 30 00:47:16.506692 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG oslo_db.sqlalchemy.engines [None req-1065301a-c3cb-44d0-8722-d91c9712f6a9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103759) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 30 00:47:16.508254 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG oslo_db.sqlalchemy.engines [None req-1065301a-c3cb-44d0-8722-d91c9712f6a9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103759) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 30 00:47:16.510988 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG dbcounter [-] [103760] Writer thread running {{(pid=103760) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 30 00:47:16.513958 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG dbcounter [-] [103760] Writer thread running {{(pid=103760) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 30 00:47:16.514129 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG dbcounter [-] [103759] Writer thread running {{(pid=103759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 30 00:47:16.516793 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG dbcounter [-] [103759] Writer thread running {{(pid=103759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 30 00:47:16.655759 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: INFO dbcounter [None req-a4281e05-8cd0-456c-bec0-7b735525da63 None None] Registered counter for database nova_cell0 Oct 30 00:47:16.657143 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: INFO dbcounter [None req-1065301a-c3cb-44d0-8722-d91c9712f6a9 None None] Registered counter for database nova_cell0 Oct 30 00:47:16.664245 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG oslo_db.sqlalchemy.engines [None req-a4281e05-8cd0-456c-bec0-7b735525da63 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103760) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 30 00:47:16.667932 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG oslo_db.sqlalchemy.engines [None req-1065301a-c3cb-44d0-8722-d91c9712f6a9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103759) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 30 00:47:16.669592 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG dbcounter [-] [103760] Writer thread running {{(pid=103760) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 30 00:47:16.672970 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG dbcounter [-] [103759] Writer thread running {{(pid=103759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 30 00:47:17.008212 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: WARNING keystonemiddleware.auth_token [None req-1065301a-c3cb-44d0-8722-d91c9712f6a9 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 30 00:47:17.008876 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: WARNING keystonemiddleware.auth_token [None req-a4281e05-8cd0-456c-bec0-7b735525da63 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 30 00:47:17.015557 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: WARNING keystonemiddleware.auth_token [None req-1065301a-c3cb-44d0-8722-d91c9712f6a9 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 30 00:47:17.015803 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: WARNING keystonemiddleware.auth_token [None req-a4281e05-8cd0-456c-bec0-7b735525da63 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 30 00:47:17.359692 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: WARNING keystonemiddleware.auth_token [None req-a4281e05-8cd0-456c-bec0-7b735525da63 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 30 00:47:17.360471 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: WARNING keystonemiddleware.auth_token [None req-1065301a-c3cb-44d0-8722-d91c9712f6a9 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 30 00:47:17.366582 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: WARNING keystonemiddleware.auth_token [None req-1065301a-c3cb-44d0-8722-d91c9712f6a9 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 30 00:47:17.366720 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: WARNING keystonemiddleware.auth_token [None req-a4281e05-8cd0-456c-bec0-7b735525da63 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 30 00:47:17.372071 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x558d3b9839c0 pid: 103759 (default app) Oct 30 00:47:17.372216 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x558d3b9839c0 pid: 103760 (default app) Oct 30 00:47:17.374180 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi [None req-a4281e05-8cd0-456c-bec0-7b735525da63 None None] Calling method '>' {{(pid=103760) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:47:17.375375 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: INFO nova.api.openstack.requestlog [None req-a4281e05-8cd0-456c-bec0-7b735525da63 None None] 10.222.0.16 "GET /compute/v2.1/" status: 200 len: 388 microversion: 2.1 time: 0.001753 Oct 30 00:47:17.375815 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: [pid: 103760|app: 0|req: 1/1] 10.222.0.16 () {50 vars in 788 bytes} [Sun Oct 30 00:47:17 2022] GET /compute/v2.1/ => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:48:38.551433 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi [None req-1065301a-c3cb-44d0-8722-d91c9712f6a9 None None] Calling method '>' {{(pid=103759) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:48:38.553355 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: INFO nova.api.openstack.requestlog [None req-1065301a-c3cb-44d0-8722-d91c9712f6a9 None None] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002644 Oct 30 00:48:38.553987 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: [pid: 103759|app: 0|req: 1/2] 10.222.0.16 () {54 vars in 931 bytes} [Sun Oct 30 00:48:38 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:48:38.573055 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG dbcounter [-] [103760] Writing DB stats nova_api:SELECT=3 {{(pid=103760) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:38.583076 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG dbcounter [-] [103760] Writing DB stats nova_cell0:SELECT=3 {{(pid=103760) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:38.584722 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG dbcounter [-] [103760] Writing DB stats nova_cell1:SELECT=3 {{(pid=103760) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:38.587610 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG dbcounter [-] [103760] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=103760) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:39.472559 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi [None req-3ee896dc-2885-4427-ad03-cbe4930c0c2c admin admin] Calling method '>' {{(pid=103760) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:48:39.505818 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: INFO nova.api.openstack.requestlog [None req-3ee896dc-2885-4427-ad03-cbe4930c0c2c admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 15 microversion: 2.61 time: 0.945189 Oct 30 00:48:39.506588 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: [pid: 103760|app: 0|req: 2/3] 10.222.0.16 () {60 vars in 1292 bytes} [Sun Oct 30 00:48:38 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 15 bytes in 947 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:48:42.126920 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi [None req-1065301a-c3cb-44d0-8722-d91c9712f6a9 None None] Calling method '>' {{(pid=103759) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:48:42.127963 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: INFO nova.api.openstack.requestlog [None req-1065301a-c3cb-44d0-8722-d91c9712f6a9 None None] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001436 Oct 30 00:48:42.128336 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: [pid: 103759|app: 0|req: 2/4] 10.222.0.16 () {54 vars in 931 bytes} [Sun Oct 30 00:48:42 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:48:42.599256 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi [None req-c8e35ea3-7937-4430-a1b5-c8a60ba76b29 admin admin] Action: 'create', calling method: >, body: {"flavor": {"os-flavor-access:is_public": true, "rxtx_factor": 1.0, "vcpus": 1, "id": "c1", "name": "cirros256", "swap": 0, "ram": 256, "OS-FLV-EXT-DATA:ephemeral": 0, "disk": 1}} {{(pid=103760) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:48:42.648412 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: INFO nova.api.openstack.requestlog [None req-c8e35ea3-7937-4430-a1b5-c8a60ba76b29 admin admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.61 time: 0.513469 Oct 30 00:48:42.649246 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: [pid: 103760|app: 0|req: 3/5] 10.222.0.16 () {64 vars in 1283 bytes} [Sun Oct 30 00:48:42 2022] POST /compute/v2.1/flavors => generated 407 bytes in 515 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 30 00:48:42.663167 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG dbcounter [-] [103759] Writing DB stats nova_api:SELECT=3 {{(pid=103759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:42.664025 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG dbcounter [-] [103759] Writing DB stats nova_cell0:SELECT=3 {{(pid=103759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:42.666018 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG dbcounter [-] [103759] Writing DB stats nova_cell1:SELECT=3 {{(pid=103759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:42.667041 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG dbcounter [-] [103759] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=103759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:42.871116 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi [None req-078937e7-c5b0-413c-9a3c-92f4087e2152 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103759) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:48:42.942216 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: INFO nova.api.openstack.requestlog [None req-078937e7-c5b0-413c-9a3c-92f4087e2152 admin admin] 10.222.0.16 "POST /compute/v2.1/flavors/c1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.283309 Oct 30 00:48:42.942216 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: [pid: 103759|app: 0|req: 3/6] 10.222.0.16 () {64 vars in 1336 bytes} [Sun Oct 30 00:48:42 2022] POST /compute/v2.1/flavors/c1/os-extra_specs => generated 43 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 00:48:45.561541 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi [None req-c8e35ea3-7937-4430-a1b5-c8a60ba76b29 admin admin] Calling method '>' {{(pid=103760) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:48:45.563338 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: INFO nova.api.openstack.requestlog [None req-c8e35ea3-7937-4430-a1b5-c8a60ba76b29 admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002236 Oct 30 00:48:45.564028 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: [pid: 103760|app: 0|req: 4/7] 10.222.0.16 () {54 vars in 931 bytes} [Sun Oct 30 00:48:45 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:48:46.208010 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi [None req-c1330c81-b9fc-4b42-8a67-657e75e2e3fa admin admin] Action: 'create', calling method: >, body: {"flavor": {"OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "ram": 512, "name": "ds512M", "swap": 0, "os-flavor-access:is_public": true, "vcpus": 1, "disk": 5, "id": "d1"}} {{(pid=103759) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:48:46.250019 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: INFO nova.api.openstack.requestlog [None req-c1330c81-b9fc-4b42-8a67-657e75e2e3fa admin admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.679782 Oct 30 00:48:46.251091 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: [pid: 103759|app: 0|req: 4/8] 10.222.0.16 () {64 vars in 1283 bytes} [Sun Oct 30 00:48:45 2022] POST /compute/v2.1/flavors => generated 404 bytes in 682 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 30 00:48:46.267834 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi [None req-6de8f24f-5a9e-4735-a822-0e0c5d28cb62 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103760) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:48:46.308909 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: INFO nova.api.openstack.requestlog [None req-6de8f24f-5a9e-4735-a822-0e0c5d28cb62 admin admin] 10.222.0.16 "POST /compute/v2.1/flavors/d1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.048971 Oct 30 00:48:46.309930 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: [pid: 103760|app: 0|req: 5/9] 10.222.0.16 () {64 vars in 1336 bytes} [Sun Oct 30 00:48:46 2022] POST /compute/v2.1/flavors/d1/os-extra_specs => generated 43 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:48:49.114148 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi [None req-c1330c81-b9fc-4b42-8a67-657e75e2e3fa admin admin] Calling method '>' {{(pid=103759) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:48:49.115266 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: INFO nova.api.openstack.requestlog [None req-c1330c81-b9fc-4b42-8a67-657e75e2e3fa admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001552 Oct 30 00:48:49.115755 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: [pid: 103759|app: 0|req: 5/10] 10.222.0.16 () {54 vars in 931 bytes} [Sun Oct 30 00:48:49 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:48:49.537354 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi [None req-92a1b9d1-b420-4cb7-8f28-b53efa3ce0bf admin admin] Action: 'create', calling method: >, body: {"flavor": {"id": "d2", "vcpus": 1, "ram": 1024, "disk": 10, "OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": true, "name": "ds1G", "swap": 0, "rxtx_factor": 1.0}} {{(pid=103760) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:48:49.560342 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: INFO nova.api.openstack.requestlog [None req-92a1b9d1-b420-4cb7-8f28-b53efa3ce0bf admin admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.437454 Oct 30 00:48:49.561075 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: [pid: 103760|app: 0|req: 6/11] 10.222.0.16 () {64 vars in 1283 bytes} [Sun Oct 30 00:48:49 2022] POST /compute/v2.1/flavors => generated 404 bytes in 439 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 30 00:48:49.574410 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi [None req-cf3d1f61-f592-4c20-9bf6-8da3d3934c07 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103759) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:48:49.603873 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: INFO nova.api.openstack.requestlog [None req-cf3d1f61-f592-4c20-9bf6-8da3d3934c07 admin admin] 10.222.0.16 "POST /compute/v2.1/flavors/d2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.033584 Oct 30 00:48:49.604797 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: [pid: 103759|app: 0|req: 6/12] 10.222.0.16 () {64 vars in 1336 bytes} [Sun Oct 30 00:48:49 2022] POST /compute/v2.1/flavors/d2/os-extra_specs => generated 43 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:48:52.274984 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi [None req-92a1b9d1-b420-4cb7-8f28-b53efa3ce0bf admin admin] Calling method '>' {{(pid=103760) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:48:52.276041 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: INFO nova.api.openstack.requestlog [None req-92a1b9d1-b420-4cb7-8f28-b53efa3ce0bf admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001470 Oct 30 00:48:52.276454 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: [pid: 103760|app: 0|req: 7/13] 10.222.0.16 () {54 vars in 931 bytes} [Sun Oct 30 00:48:52 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:48:52.720810 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi [None req-5eca5596-b019-4336-aa16-d9901f65a7da admin admin] Action: 'create', calling method: >, body: {"flavor": {"id": "d3", "rxtx_factor": 1.0, "swap": 0, "disk": 10, "OS-FLV-EXT-DATA:ephemeral": 0, "vcpus": 2, "ram": 2048, "name": "ds2G", "os-flavor-access:is_public": true}} {{(pid=103759) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:48:52.750906 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: INFO nova.api.openstack.requestlog [None req-5eca5596-b019-4336-aa16-d9901f65a7da admin admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.467782 Oct 30 00:48:52.752136 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: [pid: 103759|app: 0|req: 7/14] 10.222.0.16 () {64 vars in 1283 bytes} [Sun Oct 30 00:48:52 2022] POST /compute/v2.1/flavors => generated 404 bytes in 469 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 30 00:48:52.765799 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi [None req-ee28c136-5175-4095-a104-52bb827b81bb admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103760) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:48:52.827811 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: INFO nova.api.openstack.requestlog [None req-ee28c136-5175-4095-a104-52bb827b81bb admin admin] 10.222.0.16 "POST /compute/v2.1/flavors/d3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.067080 Oct 30 00:48:52.828672 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: [pid: 103760|app: 0|req: 8/15] 10.222.0.16 () {64 vars in 1336 bytes} [Sun Oct 30 00:48:52 2022] POST /compute/v2.1/flavors/d3/os-extra_specs => generated 43 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:48:55.449700 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi [None req-5eca5596-b019-4336-aa16-d9901f65a7da admin admin] Calling method '>' {{(pid=103759) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:48:55.450760 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: INFO nova.api.openstack.requestlog [None req-5eca5596-b019-4336-aa16-d9901f65a7da admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001568 Oct 30 00:48:55.451397 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: [pid: 103759|app: 0|req: 8/16] 10.222.0.16 () {54 vars in 931 bytes} [Sun Oct 30 00:48:55 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:48:55.917484 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi [None req-294af8ef-3571-4e0f-9042-23d4a57b6fe4 admin admin] Action: 'create', calling method: >, body: {"flavor": {"OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": true, "name": "ds4G", "rxtx_factor": 1.0, "swap": 0, "ram": 4096, "vcpus": 4, "id": "d4", "disk": 20}} {{(pid=103760) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:48:55.940214 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: INFO nova.api.openstack.requestlog [None req-294af8ef-3571-4e0f-9042-23d4a57b6fe4 admin admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.481202 Oct 30 00:48:55.940939 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: [pid: 103760|app: 0|req: 9/17] 10.222.0.16 () {64 vars in 1283 bytes} [Sun Oct 30 00:48:55 2022] POST /compute/v2.1/flavors => generated 404 bytes in 483 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 30 00:48:55.955318 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi [None req-2d561820-ba40-42d7-9ef9-c88abfe99f4d admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103759) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:48:55.987089 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: INFO nova.api.openstack.requestlog [None req-2d561820-ba40-42d7-9ef9-c88abfe99f4d admin admin] 10.222.0.16 "POST /compute/v2.1/flavors/d4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.037062 Oct 30 00:48:55.987676 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: [pid: 103759|app: 0|req: 9/18] 10.222.0.16 () {64 vars in 1336 bytes} [Sun Oct 30 00:48:55 2022] POST /compute/v2.1/flavors/d4/os-extra_specs => generated 43 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 00:48:58.762693 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi [None req-294af8ef-3571-4e0f-9042-23d4a57b6fe4 admin admin] Calling method '>' {{(pid=103760) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:48:58.763710 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: INFO nova.api.openstack.requestlog [None req-294af8ef-3571-4e0f-9042-23d4a57b6fe4 admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001494 Oct 30 00:48:58.764145 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: [pid: 103760|app: 0|req: 10/19] 10.222.0.16 () {54 vars in 931 bytes} [Sun Oct 30 00:48:58 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:48:59.200741 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi [None req-6f02daa3-e343-4fb7-8cc2-709824f7a8a6 admin admin] Calling method '>' {{(pid=103759) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:48:59.225630 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: INFO nova.api.openstack.requestlog [None req-6f02daa3-e343-4fb7-8cc2-709824f7a8a6 admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 2106 microversion: 2.61 time: 0.454694 Oct 30 00:48:59.226171 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: [pid: 103759|app: 0|req: 10/20] 10.222.0.16 () {60 vars in 1292 bytes} [Sun Oct 30 00:48:58 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 2106 bytes in 456 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 00:49:01.806485 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi [None req-294af8ef-3571-4e0f-9042-23d4a57b6fe4 admin admin] Calling method '>' {{(pid=103760) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:49:01.807507 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: INFO nova.api.openstack.requestlog [None req-294af8ef-3571-4e0f-9042-23d4a57b6fe4 admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001596 Oct 30 00:49:01.807912 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: [pid: 103760|app: 0|req: 11/21] 10.222.0.16 () {54 vars in 931 bytes} [Sun Oct 30 00:49:01 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:49:02.203003 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi [None req-c4bf73e8-445a-4966-a364-9f670d3cdc01 admin admin] Action: 'create', calling method: >, body: {"flavor": {"id": "1", "ram": 512, "os-flavor-access:is_public": true, "OS-FLV-EXT-DATA:ephemeral": 0, "disk": 1, "swap": 0, "rxtx_factor": 1.0, "name": "m1.tiny", "vcpus": 1}} {{(pid=103759) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:49:02.225079 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: INFO nova.api.openstack.requestlog [None req-c4bf73e8-445a-4966-a364-9f670d3cdc01 admin admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 402 microversion: 2.61 time: 0.410914 Oct 30 00:49:02.225718 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: [pid: 103759|app: 0|req: 11/22] 10.222.0.16 () {64 vars in 1283 bytes} [Sun Oct 30 00:49:01 2022] POST /compute/v2.1/flavors => generated 402 bytes in 412 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 30 00:49:02.238487 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi [None req-7178d622-500d-4baf-999f-cc9b4d4601cb admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103760) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:49:02.265581 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: INFO nova.api.openstack.requestlog [None req-7178d622-500d-4baf-999f-cc9b4d4601cb admin admin] 10.222.0.16 "POST /compute/v2.1/flavors/1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.031033 Oct 30 00:49:02.266190 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: [pid: 103760|app: 0|req: 12/23] 10.222.0.16 () {64 vars in 1333 bytes} [Sun Oct 30 00:49:02 2022] POST /compute/v2.1/flavors/1/os-extra_specs => generated 43 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:49:04.847029 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi [None req-c4bf73e8-445a-4966-a364-9f670d3cdc01 admin admin] Calling method '>' {{(pid=103759) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:49:04.848061 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: INFO nova.api.openstack.requestlog [None req-c4bf73e8-445a-4966-a364-9f670d3cdc01 admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001434 Oct 30 00:49:04.848470 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: [pid: 103759|app: 0|req: 12/24] 10.222.0.16 () {54 vars in 931 bytes} [Sun Oct 30 00:49:04 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:49:05.213833 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi [None req-7b81f0f0-3b01-4b4b-b072-9cbbe38fb2f1 admin admin] Action: 'create', calling method: >, body: {"flavor": {"os-flavor-access:is_public": true, "ram": 2048, "name": "m1.small", "rxtx_factor": 1.0, "disk": 20, "vcpus": 1, "OS-FLV-EXT-DATA:ephemeral": 0, "id": "2", "swap": 0}} {{(pid=103760) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:49:05.241579 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: INFO nova.api.openstack.requestlog [None req-7b81f0f0-3b01-4b4b-b072-9cbbe38fb2f1 admin admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.386760 Oct 30 00:49:05.244110 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: [pid: 103760|app: 0|req: 13/25] 10.222.0.16 () {64 vars in 1283 bytes} [Sun Oct 30 00:49:04 2022] POST /compute/v2.1/flavors => generated 405 bytes in 390 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 30 00:49:05.257057 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi [None req-c0cdb09a-ec25-4e98-806d-8148f8549311 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103759) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:49:05.288274 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: INFO nova.api.openstack.requestlog [None req-c0cdb09a-ec25-4e98-806d-8148f8549311 admin admin] 10.222.0.16 "POST /compute/v2.1/flavors/2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.034990 Oct 30 00:49:05.293216 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: [pid: 103759|app: 0|req: 13/26] 10.222.0.16 () {64 vars in 1333 bytes} [Sun Oct 30 00:49:05 2022] POST /compute/v2.1/flavors/2/os-extra_specs => generated 43 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:49:07.955078 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi [None req-7b81f0f0-3b01-4b4b-b072-9cbbe38fb2f1 admin admin] Calling method '>' {{(pid=103760) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:49:07.956065 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: INFO nova.api.openstack.requestlog [None req-7b81f0f0-3b01-4b4b-b072-9cbbe38fb2f1 admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001492 Oct 30 00:49:07.956477 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: [pid: 103760|app: 0|req: 14/27] 10.222.0.16 () {54 vars in 931 bytes} [Sun Oct 30 00:49:07 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:49:08.412429 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi [None req-b6a8d5c6-c87c-48c0-b6b2-c641a29dd17f admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 4096, "name": "m1.medium", "id": "3", "rxtx_factor": 1.0, "swap": 0, "disk": 40, "vcpus": 2, "os-flavor-access:is_public": true, "OS-FLV-EXT-DATA:ephemeral": 0}} {{(pid=103759) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:49:08.433627 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: INFO nova.api.openstack.requestlog [None req-b6a8d5c6-c87c-48c0-b6b2-c641a29dd17f admin admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 406 microversion: 2.61 time: 0.470607 Oct 30 00:49:08.434399 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: [pid: 103759|app: 0|req: 14/28] 10.222.0.16 () {64 vars in 1283 bytes} [Sun Oct 30 00:49:07 2022] POST /compute/v2.1/flavors => generated 406 bytes in 472 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 30 00:49:08.446420 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi [None req-d870c8ae-b53a-4025-9b29-75a8c2b75aec admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103760) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:49:08.474744 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: INFO nova.api.openstack.requestlog [None req-d870c8ae-b53a-4025-9b29-75a8c2b75aec admin admin] 10.222.0.16 "POST /compute/v2.1/flavors/3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.031977 Oct 30 00:49:08.475327 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: [pid: 103760|app: 0|req: 15/29] 10.222.0.16 () {64 vars in 1333 bytes} [Sun Oct 30 00:49:08 2022] POST /compute/v2.1/flavors/3/os-extra_specs => generated 43 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 00:49:11.137373 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi [None req-b6a8d5c6-c87c-48c0-b6b2-c641a29dd17f admin admin] Calling method '>' {{(pid=103759) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:49:11.138396 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: INFO nova.api.openstack.requestlog [None req-b6a8d5c6-c87c-48c0-b6b2-c641a29dd17f admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001767 Oct 30 00:49:11.140513 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: [pid: 103759|app: 0|req: 15/30] 10.222.0.16 () {54 vars in 931 bytes} [Sun Oct 30 00:49:11 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:49:11.581282 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi [None req-522b2aa4-6dad-47b9-a16e-04cd58ab20be admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.large", "ram": 8192, "swap": 0, "rxtx_factor": 1.0, "OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": true, "vcpus": 4, "disk": 80, "id": "4"}} {{(pid=103760) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:49:11.586934 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG dbcounter [-] [103760] Writing DB stats nova_api:SELECT=39,nova_api:INSERT=8,nova_api:SAVEPOINT=4,nova_api:RELEASE=4 {{(pid=103760) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:11.605829 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: INFO nova.api.openstack.requestlog [None req-522b2aa4-6dad-47b9-a16e-04cd58ab20be admin admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.460232 Oct 30 00:49:11.606526 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: [pid: 103760|app: 0|req: 16/31] 10.222.0.16 () {64 vars in 1283 bytes} [Sun Oct 30 00:49:11 2022] POST /compute/v2.1/flavors => generated 405 bytes in 462 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 30 00:49:11.622545 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi [None req-2a5029ce-6896-476a-a216-f0095abf87b3 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103759) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:49:11.646884 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: INFO nova.api.openstack.requestlog [None req-2a5029ce-6896-476a-a216-f0095abf87b3 admin admin] 10.222.0.16 "POST /compute/v2.1/flavors/4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.032043 Oct 30 00:49:11.647755 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: [pid: 103759|app: 0|req: 16/32] 10.222.0.16 () {64 vars in 1333 bytes} [Sun Oct 30 00:49:11 2022] POST /compute/v2.1/flavors/4/os-extra_specs => generated 43 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:49:14.328473 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi [None req-522b2aa4-6dad-47b9-a16e-04cd58ab20be admin admin] Calling method '>' {{(pid=103760) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:49:14.330191 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: INFO nova.api.openstack.requestlog [None req-522b2aa4-6dad-47b9-a16e-04cd58ab20be admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002205 Oct 30 00:49:14.330806 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: [pid: 103760|app: 0|req: 17/33] 10.222.0.16 () {54 vars in 931 bytes} [Sun Oct 30 00:49:14 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:49:14.773275 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi [None req-721e3370-1e10-4292-a17e-376022f4bef5 admin admin] Action: 'create', calling method: >, body: {"flavor": {"vcpus": 8, "ram": 16384, "disk": 160, "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0, "os-flavor-access:is_public": true, "name": "m1.xlarge", "rxtx_factor": 1.0, "id": "5"}} {{(pid=103759) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:49:14.781267 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG dbcounter [-] [103759] Writing DB stats nova_api:SELECT=45,nova_api:SAVEPOINT=5,nova_api:INSERT=9,nova_api:RELEASE=5 {{(pid=103759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:14.797700 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: INFO nova.api.openstack.requestlog [None req-721e3370-1e10-4292-a17e-376022f4bef5 admin admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 408 microversion: 2.61 time: 0.460001 Oct 30 00:49:14.797700 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: [pid: 103759|app: 0|req: 17/34] 10.222.0.16 () {64 vars in 1283 bytes} [Sun Oct 30 00:49:14 2022] POST /compute/v2.1/flavors => generated 408 bytes in 461 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 30 00:49:14.811097 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi [None req-d843ecf4-a18d-41b3-af5c-fad88e43f8b8 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103760) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:49:14.839109 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: INFO nova.api.openstack.requestlog [None req-d843ecf4-a18d-41b3-af5c-fad88e43f8b8 admin admin] 10.222.0.16 "POST /compute/v2.1/flavors/5/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.032797 Oct 30 00:49:14.839904 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: [pid: 103760|app: 0|req: 18/35] 10.222.0.16 () {64 vars in 1333 bytes} [Sun Oct 30 00:49:14 2022] POST /compute/v2.1/flavors/5/os-extra_specs => generated 43 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:49:31.496192 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi [None req-721e3370-1e10-4292-a17e-376022f4bef5 admin admin] Calling method '>' {{(pid=103759) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:49:31.497717 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: INFO nova.api.openstack.requestlog [None req-721e3370-1e10-4292-a17e-376022f4bef5 admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002503 Oct 30 00:49:31.498393 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: [pid: 103759|app: 0|req: 18/36] 10.222.0.16 () {54 vars in 931 bytes} [Sun Oct 30 00:49:31 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:49:31.514597 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG dbcounter [-] [103760] Writing DB stats nova_api:INSERT=2,nova_api:SELECT=8,nova_api:SAVEPOINT=1,nova_api:RELEASE=1 {{(pid=103760) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:31.932419 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi [None req-c9b34033-fbbd-4186-905b-ff994a66563c admin admin] Calling method '>' {{(pid=103760) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:49:31.951663 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: INFO nova.api.openstack.requestlog [None req-c9b34033-fbbd-4186-905b-ff994a66563c admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 4202 microversion: 2.61 time: 0.443468 Oct 30 00:49:31.952247 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: [pid: 103760|app: 0|req: 19/37] 10.222.0.16 () {60 vars in 1292 bytes} [Sun Oct 30 00:49:31 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 4202 bytes in 445 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 00:49:33.265993 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi [None req-721e3370-1e10-4292-a17e-376022f4bef5 admin admin] Calling method '>' {{(pid=103759) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:49:33.267012 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: INFO nova.api.openstack.requestlog [None req-721e3370-1e10-4292-a17e-376022f4bef5 admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001434 Oct 30 00:49:33.267403 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: [pid: 103759|app: 0|req: 19/38] 10.222.0.16 () {56 vars in 1074 bytes} [Sun Oct 30 00:49:33 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:49:33.814181 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi [None req-611f54eb-e484-4ed2-87ee-978d0d58c071 admin admin] Calling method '>' {{(pid=103760) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:49:33.819616 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: INFO nova.api.openstack.wsgi [None req-611f54eb-e484-4ed2-87ee-978d0d58c071 admin admin] HTTP exception thrown: Flavor 42 could not be found. Oct 30 00:49:33.822892 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi [None req-611f54eb-e484-4ed2-87ee-978d0d58c071 admin admin] Request method failure captured: Oct 30 00:49:33.822892 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Oct 30 00:49:33.822892 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: Accept: application/json Oct 30 00:49:33.822892 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: Accept-Encoding: gzip, deflate Oct 30 00:49:33.822892 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: Connection: keep-alive Oct 30 00:49:33.822892 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: Content-Length: 0 Oct 30 00:49:33.822892 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: Host: 10.222.0.16 Oct 30 00:49:33.822892 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: Openstack-Api-Version: compute 2.92 Oct 30 00:49:33.822892 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: Openstack-System-Scope: None Oct 30 00:49:33.822892 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: User-Agent: python-novaclient Oct 30 00:49:33.822892 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-Auth-Token: gAAAAABjXcodaMM-lBtt8yvY5ow67mdBp3NI5pJW65g2ldTyj-_i00MCMJVB2KUlLk35tmTCuSkUukWfhdXdbSrX0wkfNtrXdQw2dSEcybxYMSLmAojRK4ebJGYHGPvJNJGKk6RHeUsyHH1afmN2CmlfDJ7JeNYsCI0VvoQV-FD1mTxqAhRvfCc Oct 30 00:49:33.822892 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-Domain-Id: None Oct 30 00:49:33.822892 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-Domain-Name: None Oct 30 00:49:33.822892 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-Identity-Status: Confirmed Oct 30 00:49:33.822892 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-Is-Admin-Project: True Oct 30 00:49:33.822892 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-Openstack-Nova-Api-Version: 2.92 Oct 30 00:49:33.822892 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-Project-Domain-Id: default Oct 30 00:49:33.822892 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-Project-Domain-Name: Default Oct 30 00:49:33.822892 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-Project-Id: 3907972f9f6248debeeda8ecd254820e Oct 30 00:49:33.822892 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-Project-Name: admin Oct 30 00:49:33.822892 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-Role: admin,reader,member Oct 30 00:49:33.822892 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-Roles: admin,reader,member Oct 30 00:49:33.824682 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_3907972f9f6248debeeda8ecd254820e"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/3907972f9f6248debeeda8ecd254820e"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/3907972f9f6248debeeda8ecd254820e"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/3907972f9f6248debeeda8ecd254820e"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 00:49:33.824682 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-Tenant: admin Oct 30 00:49:33.824682 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-Tenant-Id: 3907972f9f6248debeeda8ecd254820e Oct 30 00:49:33.824682 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-Tenant-Name: admin Oct 30 00:49:33.824682 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-User: admin Oct 30 00:49:33.824682 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-User-Domain-Id: default Oct 30 00:49:33.824682 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-User-Domain-Name: Default Oct 30 00:49:33.824682 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-User-Id: ca6c0649e64b4d38bdd14a04f31beb73 Oct 30 00:49:33.824682 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-User-Name: admin Oct 30 00:49:33.824682 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: method: > Oct 30 00:49:33.824682 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: exception: Flavor 42 could not be found. Oct 30 00:49:33.824682 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: action_args: {'id': '42'} Oct 30 00:49:33.824682 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: {{(pid=103760) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 00:49:33.824682 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 00:49:33.825794 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 30 00:49:33.825794 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 30 00:49:33.825794 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 30 00:49:33.825794 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 30 00:49:33.825794 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 30 00:49:33.825794 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 30 00:49:33.825794 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 30 00:49:33.825794 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 30 00:49:33.825794 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 30 00:49:33.825794 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 00:49:33.825794 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 30 00:49:33.825794 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 30 00:49:33.825794 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Oct 30 00:49:33.825794 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi Oct 30 00:49:33.825794 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 00:49:33.825794 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi Oct 30 00:49:33.825794 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 00:49:33.825794 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 00:49:33.825794 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 00:49:33.827164 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 00:49:33.827164 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 00:49:33.827164 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 00:49:33.827164 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 00:49:33.827164 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 30 00:49:33.827164 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 30 00:49:33.827164 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Oct 30 00:49:33.827164 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi Oct 30 00:49:33.827164 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 00:49:33.827164 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi Oct 30 00:49:33.827164 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 00:49:33.827164 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 00:49:33.827164 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 00:49:33.827164 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 00:49:33.827164 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 00:49:33.827164 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Oct 30 00:49:33.827164 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi Oct 30 00:49:33.827164 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi [None req-611f54eb-e484-4ed2-87ee-978d0d58c071 admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=103760) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 00:49:33.828517 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: INFO nova.api.openstack.requestlog [None req-611f54eb-e484-4ed2-87ee-978d0d58c071 admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.92 time: 0.420713 Oct 30 00:49:33.828517 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: [pid: 103760|app: 0|req: 20/39] 10.222.0.16 () {60 vars in 1190 bytes} [Sun Oct 30 00:49:33 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 422 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 30 00:49:33.834953 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi [None req-392d11cc-24a7-4d31-9020-5aa53708b45d admin admin] Calling method '>' {{(pid=103759) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:49:33.857377 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: INFO nova.api.openstack.requestlog [None req-392d11cc-24a7-4d31-9020-5aa53708b45d admin admin] 10.222.0.16 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.026359 Oct 30 00:49:33.857952 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: [pid: 103759|app: 0|req: 20/40] 10.222.0.16 () {60 vars in 1210 bytes} [Sun Oct 30 00:49:33 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 27 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 00:49:33.868671 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi [None req-cd1deec8-6b5b-4175-8d5c-be599919cd19 admin admin] Calling method '>' {{(pid=103760) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:49:33.891156 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: INFO nova.api.openstack.requestlog [None req-cd1deec8-6b5b-4175-8d5c-be599919cd19 admin admin] 10.222.0.16 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.027022 Oct 30 00:49:33.891684 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: [pid: 103760|app: 0|req: 21/41] 10.222.0.16 () {60 vars in 1210 bytes} [Sun Oct 30 00:49:33 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 28 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 00:49:33.901677 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi [None req-655162ed-cf95-403e-a1fd-9fd794d1e641 admin admin] Calling method '>' {{(pid=103759) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:49:33.907482 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: INFO nova.api.openstack.wsgi [None req-655162ed-cf95-403e-a1fd-9fd794d1e641 admin admin] HTTP exception thrown: Flavor 42 could not be found. Oct 30 00:49:33.910788 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi [None req-655162ed-cf95-403e-a1fd-9fd794d1e641 admin admin] Request method failure captured: Oct 30 00:49:33.910788 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Oct 30 00:49:33.910788 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: Accept: application/json Oct 30 00:49:33.910788 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: Accept-Encoding: gzip, deflate Oct 30 00:49:33.910788 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: Connection: keep-alive Oct 30 00:49:33.910788 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: Content-Length: 0 Oct 30 00:49:33.910788 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: Host: 10.222.0.16 Oct 30 00:49:33.910788 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: Openstack-Api-Version: compute 2.92 Oct 30 00:49:33.910788 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: Openstack-System-Scope: None Oct 30 00:49:33.910788 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: User-Agent: python-novaclient Oct 30 00:49:33.910788 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-Auth-Token: gAAAAABjXcodaMM-lBtt8yvY5ow67mdBp3NI5pJW65g2ldTyj-_i00MCMJVB2KUlLk35tmTCuSkUukWfhdXdbSrX0wkfNtrXdQw2dSEcybxYMSLmAojRK4ebJGYHGPvJNJGKk6RHeUsyHH1afmN2CmlfDJ7JeNYsCI0VvoQV-FD1mTxqAhRvfCc Oct 30 00:49:33.910788 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-Domain-Id: None Oct 30 00:49:33.910788 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-Domain-Name: None Oct 30 00:49:33.910788 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-Identity-Status: Confirmed Oct 30 00:49:33.910788 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-Is-Admin-Project: True Oct 30 00:49:33.910788 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-Openstack-Nova-Api-Version: 2.92 Oct 30 00:49:33.910788 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-Project-Domain-Id: default Oct 30 00:49:33.910788 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-Project-Domain-Name: Default Oct 30 00:49:33.910788 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-Project-Id: 3907972f9f6248debeeda8ecd254820e Oct 30 00:49:33.910788 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-Project-Name: admin Oct 30 00:49:33.910788 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-Role: admin,reader,member Oct 30 00:49:33.910788 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-Roles: admin,reader,member Oct 30 00:49:33.912387 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_3907972f9f6248debeeda8ecd254820e"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/3907972f9f6248debeeda8ecd254820e"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/3907972f9f6248debeeda8ecd254820e"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/3907972f9f6248debeeda8ecd254820e"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 00:49:33.912387 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-Tenant: admin Oct 30 00:49:33.912387 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-Tenant-Id: 3907972f9f6248debeeda8ecd254820e Oct 30 00:49:33.912387 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-Tenant-Name: admin Oct 30 00:49:33.912387 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-User: admin Oct 30 00:49:33.912387 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-User-Domain-Id: default Oct 30 00:49:33.912387 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-User-Domain-Name: Default Oct 30 00:49:33.912387 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-User-Id: ca6c0649e64b4d38bdd14a04f31beb73 Oct 30 00:49:33.912387 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-User-Name: admin Oct 30 00:49:33.912387 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: method: > Oct 30 00:49:33.912387 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: exception: Flavor 42 could not be found. Oct 30 00:49:33.912387 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: action_args: {'id': '42'} Oct 30 00:49:33.912387 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: {{(pid=103759) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 00:49:33.912387 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 00:49:33.913372 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 30 00:49:33.913372 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 30 00:49:33.913372 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 30 00:49:33.913372 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 30 00:49:33.913372 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 30 00:49:33.913372 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 30 00:49:33.913372 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 30 00:49:33.913372 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 30 00:49:33.913372 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 30 00:49:33.913372 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 00:49:33.913372 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 30 00:49:33.913372 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 30 00:49:33.913372 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Oct 30 00:49:33.913372 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi Oct 30 00:49:33.913372 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 00:49:33.913372 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi Oct 30 00:49:33.913372 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 00:49:33.913372 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 00:49:33.913372 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 00:49:33.914510 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 00:49:33.914510 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 00:49:33.914510 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 00:49:33.914510 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 00:49:33.914510 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 30 00:49:33.914510 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 30 00:49:33.914510 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Oct 30 00:49:33.914510 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi Oct 30 00:49:33.914510 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 00:49:33.914510 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi Oct 30 00:49:33.914510 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 00:49:33.914510 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 00:49:33.914510 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 00:49:33.914510 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 00:49:33.914510 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 00:49:33.914510 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Oct 30 00:49:33.914510 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi Oct 30 00:49:33.914510 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi [None req-655162ed-cf95-403e-a1fd-9fd794d1e641 admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=103759) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 00:49:33.915598 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: INFO nova.api.openstack.requestlog [None req-655162ed-cf95-403e-a1fd-9fd794d1e641 admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.92 time: 0.014500 Oct 30 00:49:33.915598 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: [pid: 103759|app: 0|req: 21/42] 10.222.0.16 () {60 vars in 1190 bytes} [Sun Oct 30 00:49:33 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 15 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 30 00:49:35.123538 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi [None req-cd1deec8-6b5b-4175-8d5c-be599919cd19 admin admin] Calling method '>' {{(pid=103760) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:49:35.124582 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: INFO nova.api.openstack.requestlog [None req-cd1deec8-6b5b-4175-8d5c-be599919cd19 admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001489 Oct 30 00:49:35.125020 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: [pid: 103760|app: 0|req: 22/43] 10.222.0.16 () {56 vars in 1074 bytes} [Sun Oct 30 00:49:35 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:49:35.666052 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi [None req-32969170-e2e4-4dee-b800-52ec2f38549d admin admin] Calling method '>' {{(pid=103759) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:49:35.671547 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: INFO nova.api.openstack.wsgi [None req-32969170-e2e4-4dee-b800-52ec2f38549d admin admin] HTTP exception thrown: Flavor 84 could not be found. Oct 30 00:49:35.673109 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi [None req-32969170-e2e4-4dee-b800-52ec2f38549d admin admin] Request method failure captured: Oct 30 00:49:35.673109 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Oct 30 00:49:35.673109 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: Accept: application/json Oct 30 00:49:35.673109 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: Accept-Encoding: gzip, deflate Oct 30 00:49:35.673109 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: Connection: keep-alive Oct 30 00:49:35.673109 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: Content-Length: 0 Oct 30 00:49:35.673109 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: Host: 10.222.0.16 Oct 30 00:49:35.673109 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: Openstack-Api-Version: compute 2.92 Oct 30 00:49:35.673109 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: Openstack-System-Scope: None Oct 30 00:49:35.673109 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: User-Agent: python-novaclient Oct 30 00:49:35.673109 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-Auth-Token: gAAAAABjXcofeVmTyt4uiy8Q7IgRV-4XS6iNed0YxwSR5yO99cLcsak5-2txxMq7N04Fm7IhmfhdqlyDY6nw1prWHYul5OGz_M7TKw3VeD9cMzFss-Vpz8CYrd7mYOBWDO3n0gNiTkrVh9XNVS94VBq39ta8mSY2vP7tbGcjh-6OD3m5a7D1Emc Oct 30 00:49:35.673109 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-Domain-Id: None Oct 30 00:49:35.673109 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-Domain-Name: None Oct 30 00:49:35.673109 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-Identity-Status: Confirmed Oct 30 00:49:35.673109 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-Is-Admin-Project: True Oct 30 00:49:35.673109 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-Openstack-Nova-Api-Version: 2.92 Oct 30 00:49:35.673109 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-Project-Domain-Id: default Oct 30 00:49:35.673109 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-Project-Domain-Name: Default Oct 30 00:49:35.673109 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-Project-Id: 3907972f9f6248debeeda8ecd254820e Oct 30 00:49:35.673109 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-Project-Name: admin Oct 30 00:49:35.673109 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-Role: admin,reader,member Oct 30 00:49:35.673109 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-Roles: admin,reader,member Oct 30 00:49:35.674658 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_3907972f9f6248debeeda8ecd254820e"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/3907972f9f6248debeeda8ecd254820e"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/3907972f9f6248debeeda8ecd254820e"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/3907972f9f6248debeeda8ecd254820e"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 00:49:35.674658 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-Tenant: admin Oct 30 00:49:35.674658 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-Tenant-Id: 3907972f9f6248debeeda8ecd254820e Oct 30 00:49:35.674658 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-Tenant-Name: admin Oct 30 00:49:35.674658 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-User: admin Oct 30 00:49:35.674658 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-User-Domain-Id: default Oct 30 00:49:35.674658 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-User-Domain-Name: Default Oct 30 00:49:35.674658 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-User-Id: ca6c0649e64b4d38bdd14a04f31beb73 Oct 30 00:49:35.674658 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-User-Name: admin Oct 30 00:49:35.674658 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: method: > Oct 30 00:49:35.674658 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: exception: Flavor 84 could not be found. Oct 30 00:49:35.674658 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: action_args: {'id': '84'} Oct 30 00:49:35.674658 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: {{(pid=103759) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 00:49:35.674658 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 00:49:35.675746 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 30 00:49:35.675746 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 30 00:49:35.675746 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 30 00:49:35.675746 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 30 00:49:35.675746 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 30 00:49:35.675746 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 30 00:49:35.675746 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 30 00:49:35.675746 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 30 00:49:35.675746 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 30 00:49:35.675746 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 00:49:35.675746 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 30 00:49:35.675746 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 30 00:49:35.675746 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Oct 30 00:49:35.675746 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi Oct 30 00:49:35.675746 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 00:49:35.675746 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi Oct 30 00:49:35.675746 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 00:49:35.675746 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 00:49:35.675746 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 00:49:35.677273 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 00:49:35.677273 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 00:49:35.677273 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 00:49:35.677273 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 00:49:35.677273 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 30 00:49:35.677273 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 30 00:49:35.677273 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Oct 30 00:49:35.677273 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi Oct 30 00:49:35.677273 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 00:49:35.677273 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi Oct 30 00:49:35.677273 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 00:49:35.677273 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 00:49:35.677273 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 00:49:35.677273 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 00:49:35.677273 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 00:49:35.677273 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Oct 30 00:49:35.677273 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi Oct 30 00:49:35.677273 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi [None req-32969170-e2e4-4dee-b800-52ec2f38549d admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=103759) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 00:49:35.678581 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: INFO nova.api.openstack.requestlog [None req-32969170-e2e4-4dee-b800-52ec2f38549d admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.92 time: 0.408358 Oct 30 00:49:35.678581 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: [pid: 103759|app: 0|req: 22/44] 10.222.0.16 () {60 vars in 1190 bytes} [Sun Oct 30 00:49:35 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 409 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 30 00:49:35.684324 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi [None req-dfe51e82-c12e-4a01-8086-0bf0ee3cf7dc admin admin] Calling method '>' {{(pid=103760) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:49:35.699510 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: INFO nova.api.openstack.requestlog [None req-dfe51e82-c12e-4a01-8086-0bf0ee3cf7dc admin admin] 10.222.0.16 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.018450 Oct 30 00:49:35.700076 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: [pid: 103760|app: 0|req: 23/45] 10.222.0.16 () {60 vars in 1210 bytes} [Sun Oct 30 00:49:35 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 30 00:49:35.709577 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi [None req-eede22dd-2fb9-46a5-a81d-16081b7f831e admin admin] Calling method '>' {{(pid=103759) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:49:35.724993 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: INFO nova.api.openstack.requestlog [None req-eede22dd-2fb9-46a5-a81d-16081b7f831e admin admin] 10.222.0.16 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.018973 Oct 30 00:49:35.725509 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: [pid: 103759|app: 0|req: 23/46] 10.222.0.16 () {60 vars in 1210 bytes} [Sun Oct 30 00:49:35 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 00:49:35.734677 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi [None req-c9aa8955-5b1c-4217-8b8e-5f3cf72947bd admin admin] Calling method '>' {{(pid=103760) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:49:35.739587 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: INFO nova.api.openstack.wsgi [None req-c9aa8955-5b1c-4217-8b8e-5f3cf72947bd admin admin] HTTP exception thrown: Flavor 84 could not be found. Oct 30 00:49:35.741117 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi [None req-c9aa8955-5b1c-4217-8b8e-5f3cf72947bd admin admin] Request method failure captured: Oct 30 00:49:35.741117 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Oct 30 00:49:35.741117 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: Accept: application/json Oct 30 00:49:35.741117 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: Accept-Encoding: gzip, deflate Oct 30 00:49:35.741117 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: Connection: keep-alive Oct 30 00:49:35.741117 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: Content-Length: 0 Oct 30 00:49:35.741117 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: Host: 10.222.0.16 Oct 30 00:49:35.741117 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: Openstack-Api-Version: compute 2.92 Oct 30 00:49:35.741117 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: Openstack-System-Scope: None Oct 30 00:49:35.741117 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: User-Agent: python-novaclient Oct 30 00:49:35.741117 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-Auth-Token: gAAAAABjXcofeVmTyt4uiy8Q7IgRV-4XS6iNed0YxwSR5yO99cLcsak5-2txxMq7N04Fm7IhmfhdqlyDY6nw1prWHYul5OGz_M7TKw3VeD9cMzFss-Vpz8CYrd7mYOBWDO3n0gNiTkrVh9XNVS94VBq39ta8mSY2vP7tbGcjh-6OD3m5a7D1Emc Oct 30 00:49:35.741117 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-Domain-Id: None Oct 30 00:49:35.741117 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-Domain-Name: None Oct 30 00:49:35.741117 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-Identity-Status: Confirmed Oct 30 00:49:35.741117 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-Is-Admin-Project: True Oct 30 00:49:35.741117 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-Openstack-Nova-Api-Version: 2.92 Oct 30 00:49:35.741117 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-Project-Domain-Id: default Oct 30 00:49:35.741117 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-Project-Domain-Name: Default Oct 30 00:49:35.741117 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-Project-Id: 3907972f9f6248debeeda8ecd254820e Oct 30 00:49:35.741117 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-Project-Name: admin Oct 30 00:49:35.741117 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-Role: admin,reader,member Oct 30 00:49:35.741117 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-Roles: admin,reader,member Oct 30 00:49:35.742481 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_3907972f9f6248debeeda8ecd254820e"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/3907972f9f6248debeeda8ecd254820e"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/3907972f9f6248debeeda8ecd254820e"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/3907972f9f6248debeeda8ecd254820e"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 00:49:35.742481 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-Tenant: admin Oct 30 00:49:35.742481 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-Tenant-Id: 3907972f9f6248debeeda8ecd254820e Oct 30 00:49:35.742481 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-Tenant-Name: admin Oct 30 00:49:35.742481 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-User: admin Oct 30 00:49:35.742481 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-User-Domain-Id: default Oct 30 00:49:35.742481 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-User-Domain-Name: Default Oct 30 00:49:35.742481 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-User-Id: ca6c0649e64b4d38bdd14a04f31beb73 Oct 30 00:49:35.742481 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-User-Name: admin Oct 30 00:49:35.742481 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: method: > Oct 30 00:49:35.742481 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: exception: Flavor 84 could not be found. Oct 30 00:49:35.742481 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: action_args: {'id': '84'} Oct 30 00:49:35.742481 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: {{(pid=103760) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 00:49:35.742481 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 00:49:35.743266 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 30 00:49:35.743266 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 30 00:49:35.743266 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 30 00:49:35.743266 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 30 00:49:35.743266 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 30 00:49:35.743266 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 30 00:49:35.743266 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 30 00:49:35.743266 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 30 00:49:35.743266 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 30 00:49:35.743266 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 00:49:35.743266 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 30 00:49:35.743266 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 30 00:49:35.743266 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Oct 30 00:49:35.743266 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi Oct 30 00:49:35.743266 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 00:49:35.743266 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi Oct 30 00:49:35.743266 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 00:49:35.743266 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 00:49:35.743266 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 00:49:35.744429 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 00:49:35.744429 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 00:49:35.744429 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 00:49:35.744429 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 00:49:35.744429 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 30 00:49:35.744429 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 30 00:49:35.744429 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Oct 30 00:49:35.744429 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi Oct 30 00:49:35.744429 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 00:49:35.744429 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi Oct 30 00:49:35.744429 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 00:49:35.744429 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 00:49:35.744429 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 00:49:35.744429 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 00:49:35.744429 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 00:49:35.744429 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Oct 30 00:49:35.744429 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi Oct 30 00:49:35.744429 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi [None req-c9aa8955-5b1c-4217-8b8e-5f3cf72947bd admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=103760) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 00:49:35.745393 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: INFO nova.api.openstack.requestlog [None req-c9aa8955-5b1c-4217-8b8e-5f3cf72947bd admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.92 time: 0.011358 Oct 30 00:49:35.745393 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: [pid: 103760|app: 0|req: 24/47] 10.222.0.16 () {60 vars in 1190 bytes} [Sun Oct 30 00:49:35 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 12 msecs (HTTP/1.1 404) 9 headers in 380 bytes (2 switches on core 0) Oct 30 00:49:36.990391 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi [None req-eede22dd-2fb9-46a5-a81d-16081b7f831e admin admin] Calling method '>' {{(pid=103759) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:49:36.991712 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: INFO nova.api.openstack.requestlog [None req-eede22dd-2fb9-46a5-a81d-16081b7f831e admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001971 Oct 30 00:49:36.992270 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: [pid: 103759|app: 0|req: 24/48] 10.222.0.16 () {56 vars in 1074 bytes} [Sun Oct 30 00:49:36 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:49:37.545862 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi [None req-522c2a71-21b1-4525-b513-779e78e228db admin admin] Calling method '>' {{(pid=103760) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:49:37.552147 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: INFO nova.api.openstack.wsgi [None req-522c2a71-21b1-4525-b513-779e78e228db admin admin] HTTP exception thrown: Flavor 451 could not be found. Oct 30 00:49:37.553637 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi [None req-522c2a71-21b1-4525-b513-779e78e228db admin admin] Request method failure captured: Oct 30 00:49:37.553637 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Oct 30 00:49:37.553637 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: Accept: application/json Oct 30 00:49:37.553637 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: Accept-Encoding: gzip, deflate Oct 30 00:49:37.553637 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: Connection: keep-alive Oct 30 00:49:37.553637 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: Content-Length: 0 Oct 30 00:49:37.553637 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: Host: 10.222.0.16 Oct 30 00:49:37.553637 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: Openstack-Api-Version: compute 2.92 Oct 30 00:49:37.553637 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: Openstack-System-Scope: None Oct 30 00:49:37.553637 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: User-Agent: python-novaclient Oct 30 00:49:37.553637 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-Auth-Token: gAAAAABjXcog7DSp7eKl4Uwjs5WC4qpzMoJFZEOkwdgFBZGWm_VA119HbzMNqLJFbfvwwNrL5Kz42Kv9_ZgRYuJCs9v-LogMjrq0vW4xQvFjO1evgTR25SmndH4xIgB2oekalXZOHPImmDTSo00juR873LULlvisaqPLE0QJD2d1R3w8pG5rmTs Oct 30 00:49:37.553637 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-Domain-Id: None Oct 30 00:49:37.553637 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-Domain-Name: None Oct 30 00:49:37.553637 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-Identity-Status: Confirmed Oct 30 00:49:37.553637 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-Is-Admin-Project: True Oct 30 00:49:37.553637 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-Openstack-Nova-Api-Version: 2.92 Oct 30 00:49:37.553637 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-Project-Domain-Id: default Oct 30 00:49:37.553637 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-Project-Domain-Name: Default Oct 30 00:49:37.553637 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-Project-Id: 3907972f9f6248debeeda8ecd254820e Oct 30 00:49:37.553637 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-Project-Name: admin Oct 30 00:49:37.553637 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-Role: admin,reader,member Oct 30 00:49:37.553637 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-Roles: admin,reader,member Oct 30 00:49:37.555087 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_3907972f9f6248debeeda8ecd254820e"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/3907972f9f6248debeeda8ecd254820e"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/3907972f9f6248debeeda8ecd254820e"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/3907972f9f6248debeeda8ecd254820e"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 00:49:37.555087 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-Tenant: admin Oct 30 00:49:37.555087 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-Tenant-Id: 3907972f9f6248debeeda8ecd254820e Oct 30 00:49:37.555087 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-Tenant-Name: admin Oct 30 00:49:37.555087 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-User: admin Oct 30 00:49:37.555087 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-User-Domain-Id: default Oct 30 00:49:37.555087 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-User-Domain-Name: Default Oct 30 00:49:37.555087 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-User-Id: ca6c0649e64b4d38bdd14a04f31beb73 Oct 30 00:49:37.555087 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: X-User-Name: admin Oct 30 00:49:37.555087 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: method: > Oct 30 00:49:37.555087 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: exception: Flavor 451 could not be found. Oct 30 00:49:37.555087 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: action_args: {'id': '451'} Oct 30 00:49:37.555087 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: {{(pid=103760) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 00:49:37.555087 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 00:49:37.560456 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 30 00:49:37.560456 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 30 00:49:37.560456 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 30 00:49:37.560456 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 30 00:49:37.560456 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 30 00:49:37.560456 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 30 00:49:37.560456 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 30 00:49:37.560456 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 30 00:49:37.560456 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 30 00:49:37.560456 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 00:49:37.560456 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 30 00:49:37.560456 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 30 00:49:37.560456 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Oct 30 00:49:37.560456 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi Oct 30 00:49:37.560456 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 00:49:37.560456 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi Oct 30 00:49:37.560456 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 00:49:37.560456 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 00:49:37.560456 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 00:49:37.561862 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 00:49:37.561862 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 00:49:37.561862 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 00:49:37.561862 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 00:49:37.561862 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 30 00:49:37.561862 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 30 00:49:37.561862 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Oct 30 00:49:37.561862 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi Oct 30 00:49:37.561862 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 00:49:37.561862 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi Oct 30 00:49:37.561862 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 00:49:37.561862 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 00:49:37.561862 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 00:49:37.561862 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 00:49:37.561862 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 00:49:37.561862 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Oct 30 00:49:37.561862 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: ERROR nova.api.openstack.wsgi Oct 30 00:49:37.561862 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi [None req-522c2a71-21b1-4525-b513-779e78e228db admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=103760) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 00:49:37.563090 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: INFO nova.api.openstack.requestlog [None req-522c2a71-21b1-4525-b513-779e78e228db admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.92 time: 0.424956 Oct 30 00:49:37.563090 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: [pid: 103760|app: 0|req: 25/49] 10.222.0.16 () {60 vars in 1193 bytes} [Sun Oct 30 00:49:37 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 426 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 30 00:49:37.566276 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi [None req-1c1d893c-ac9f-402d-8ef8-c9644a228974 admin admin] Calling method '>' {{(pid=103759) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:49:37.581362 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: INFO nova.api.openstack.requestlog [None req-1c1d893c-ac9f-402d-8ef8-c9644a228974 admin admin] 10.222.0.16 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.018703 Oct 30 00:49:37.581885 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: [pid: 103759|app: 0|req: 25/50] 10.222.0.16 () {60 vars in 1210 bytes} [Sun Oct 30 00:49:37 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 00:49:37.592811 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi [None req-0549f73d-759d-4f38-9280-068363435135 admin admin] Calling method '>' {{(pid=103760) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:49:37.608288 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: INFO nova.api.openstack.requestlog [None req-0549f73d-759d-4f38-9280-068363435135 admin admin] 10.222.0.16 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.020060 Oct 30 00:49:37.608840 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: [pid: 103760|app: 0|req: 26/51] 10.222.0.16 () {60 vars in 1210 bytes} [Sun Oct 30 00:49:37 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 21 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 00:49:37.618138 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi [None req-d0e0050f-ee1b-4307-a580-cf0ec0153824 admin admin] Calling method '>' {{(pid=103759) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:49:37.623219 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: INFO nova.api.openstack.wsgi [None req-d0e0050f-ee1b-4307-a580-cf0ec0153824 admin admin] HTTP exception thrown: Flavor 451 could not be found. Oct 30 00:49:37.624775 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi [None req-d0e0050f-ee1b-4307-a580-cf0ec0153824 admin admin] Request method failure captured: Oct 30 00:49:37.624775 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Oct 30 00:49:37.624775 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: Accept: application/json Oct 30 00:49:37.624775 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: Accept-Encoding: gzip, deflate Oct 30 00:49:37.624775 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: Connection: keep-alive Oct 30 00:49:37.624775 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: Content-Length: 0 Oct 30 00:49:37.624775 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: Host: 10.222.0.16 Oct 30 00:49:37.624775 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: Openstack-Api-Version: compute 2.92 Oct 30 00:49:37.624775 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: Openstack-System-Scope: None Oct 30 00:49:37.624775 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: User-Agent: python-novaclient Oct 30 00:49:37.624775 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-Auth-Token: gAAAAABjXcog7DSp7eKl4Uwjs5WC4qpzMoJFZEOkwdgFBZGWm_VA119HbzMNqLJFbfvwwNrL5Kz42Kv9_ZgRYuJCs9v-LogMjrq0vW4xQvFjO1evgTR25SmndH4xIgB2oekalXZOHPImmDTSo00juR873LULlvisaqPLE0QJD2d1R3w8pG5rmTs Oct 30 00:49:37.624775 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-Domain-Id: None Oct 30 00:49:37.624775 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-Domain-Name: None Oct 30 00:49:37.624775 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-Identity-Status: Confirmed Oct 30 00:49:37.624775 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-Is-Admin-Project: True Oct 30 00:49:37.624775 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-Openstack-Nova-Api-Version: 2.92 Oct 30 00:49:37.624775 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-Project-Domain-Id: default Oct 30 00:49:37.624775 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-Project-Domain-Name: Default Oct 30 00:49:37.624775 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-Project-Id: 3907972f9f6248debeeda8ecd254820e Oct 30 00:49:37.624775 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-Project-Name: admin Oct 30 00:49:37.624775 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-Role: admin,reader,member Oct 30 00:49:37.624775 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-Roles: admin,reader,member Oct 30 00:49:37.626260 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_3907972f9f6248debeeda8ecd254820e"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/3907972f9f6248debeeda8ecd254820e"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/3907972f9f6248debeeda8ecd254820e"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/3907972f9f6248debeeda8ecd254820e"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 00:49:37.626260 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-Tenant: admin Oct 30 00:49:37.626260 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-Tenant-Id: 3907972f9f6248debeeda8ecd254820e Oct 30 00:49:37.626260 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-Tenant-Name: admin Oct 30 00:49:37.626260 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-User: admin Oct 30 00:49:37.626260 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-User-Domain-Id: default Oct 30 00:49:37.626260 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-User-Domain-Name: Default Oct 30 00:49:37.626260 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-User-Id: ca6c0649e64b4d38bdd14a04f31beb73 Oct 30 00:49:37.626260 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: X-User-Name: admin Oct 30 00:49:37.626260 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: method: > Oct 30 00:49:37.626260 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: exception: Flavor 451 could not be found. Oct 30 00:49:37.626260 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: action_args: {'id': '451'} Oct 30 00:49:37.626260 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: {{(pid=103759) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 00:49:37.626260 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 00:49:37.627198 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 30 00:49:37.627198 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 30 00:49:37.627198 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 30 00:49:37.627198 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 30 00:49:37.627198 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 30 00:49:37.627198 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 30 00:49:37.627198 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 30 00:49:37.627198 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 30 00:49:37.627198 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 30 00:49:37.627198 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 00:49:37.627198 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 30 00:49:37.627198 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 30 00:49:37.627198 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Oct 30 00:49:37.627198 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi Oct 30 00:49:37.627198 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 00:49:37.627198 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi Oct 30 00:49:37.627198 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 00:49:37.627198 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 00:49:37.627198 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 00:49:37.628512 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 00:49:37.628512 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 00:49:37.628512 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 00:49:37.628512 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 00:49:37.628512 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 30 00:49:37.628512 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 30 00:49:37.628512 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Oct 30 00:49:37.628512 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi Oct 30 00:49:37.628512 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 00:49:37.628512 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi Oct 30 00:49:37.628512 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 00:49:37.628512 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 00:49:37.628512 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 00:49:37.628512 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 00:49:37.628512 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 00:49:37.628512 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Oct 30 00:49:37.628512 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: ERROR nova.api.openstack.wsgi Oct 30 00:49:37.628512 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi [None req-d0e0050f-ee1b-4307-a580-cf0ec0153824 admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=103759) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 00:49:37.629739 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: INFO nova.api.openstack.requestlog [None req-d0e0050f-ee1b-4307-a580-cf0ec0153824 admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.92 time: 0.011495 Oct 30 00:49:37.629739 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: [pid: 103759|app: 0|req: 26/52] 10.222.0.16 () {60 vars in 1193 bytes} [Sun Oct 30 00:49:37 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 12 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 30 00:49:38.848090 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi [None req-0549f73d-759d-4f38-9280-068363435135 admin admin] Calling method '>' {{(pid=103760) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:49:38.849108 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: INFO nova.api.openstack.requestlog [None req-0549f73d-759d-4f38-9280-068363435135 admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001516 Oct 30 00:49:38.849552 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: [pid: 103760|app: 0|req: 27/53] 10.222.0.16 () {56 vars in 1074 bytes} [Sun Oct 30 00:49:38 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:49:39.402429 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi [None req-ea8d5c5b-9107-4755-89f0-b56d4d1d0aa5 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.nano", "ram": 256, "vcpus": 1, "disk": 1, "id": "42", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103759) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:49:39.425225 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: INFO nova.api.openstack.requestlog [None req-ea8d5c5b-9107-4755-89f0-b56d4d1d0aa5 admin admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.92 time: 0.440717 Oct 30 00:49:39.425765 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: [pid: 103759|app: 0|req: 27/54] 10.222.0.16 () {64 vars in 1235 bytes} [Sun Oct 30 00:49:38 2022] POST /compute/v2.1/flavors => generated 404 bytes in 442 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 30 00:49:40.645694 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi [None req-0549f73d-759d-4f38-9280-068363435135 admin admin] Calling method '>' {{(pid=103760) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:49:40.646701 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: INFO nova.api.openstack.requestlog [None req-0549f73d-759d-4f38-9280-068363435135 admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001445 Oct 30 00:49:40.647096 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: [pid: 103760|app: 0|req: 28/55] 10.222.0.16 () {56 vars in 1074 bytes} [Sun Oct 30 00:49:40 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:49:41.204814 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi [None req-79859f22-23dc-48e9-a3d9-eb35560c19ff admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.micro", "ram": 256, "vcpus": 1, "disk": 2, "id": "84", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103759) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:49:41.225591 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: INFO nova.api.openstack.requestlog [None req-79859f22-23dc-48e9-a3d9-eb35560c19ff admin admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.92 time: 0.442547 Oct 30 00:49:41.226160 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: [pid: 103759|app: 0|req: 28/56] 10.222.0.16 () {64 vars in 1235 bytes} [Sun Oct 30 00:49:40 2022] POST /compute/v2.1/flavors => generated 405 bytes in 444 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 30 00:49:42.440924 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi [None req-0549f73d-759d-4f38-9280-068363435135 admin admin] Calling method '>' {{(pid=103760) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:49:42.441914 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: INFO nova.api.openstack.requestlog [None req-0549f73d-759d-4f38-9280-068363435135 admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001469 Oct 30 00:49:42.442343 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: [pid: 103760|app: 0|req: 29/57] 10.222.0.16 () {56 vars in 1074 bytes} [Sun Oct 30 00:49:42 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:49:43.010802 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi [None req-c742e23a-6afc-4607-9a9c-0de37686b35c admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat", "ram": 512, "vcpus": 1, "disk": 1, "id": "451", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103759) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:49:43.036526 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: INFO nova.api.openstack.requestlog [None req-c742e23a-6afc-4607-9a9c-0de37686b35c admin admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.92 time: 0.455415 Oct 30 00:49:43.037083 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: [pid: 103759|app: 0|req: 29/58] 10.222.0.16 () {64 vars in 1235 bytes} [Sun Oct 30 00:49:42 2022] POST /compute/v2.1/flavors => generated 407 bytes in 457 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 30 00:49:44.268088 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi [None req-0549f73d-759d-4f38-9280-068363435135 admin admin] Calling method '>' {{(pid=103760) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:49:44.269103 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: INFO nova.api.openstack.requestlog [None req-0549f73d-759d-4f38-9280-068363435135 admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001499 Oct 30 00:49:44.269531 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: [pid: 103760|app: 0|req: 30/59] 10.222.0.16 () {56 vars in 1074 bytes} [Sun Oct 30 00:49:44 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:49:44.816729 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi [None req-193d73ca-8fcd-4461-bcbe-d340e1ff8b41 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat2", "ram": 1024, "vcpus": 1, "disk": 1, "id": "452", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103759) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:49:44.838220 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: INFO nova.api.openstack.requestlog [None req-193d73ca-8fcd-4461-bcbe-d340e1ff8b41 admin admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 409 microversion: 2.92 time: 0.431091 Oct 30 00:49:44.838760 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: [pid: 103759|app: 0|req: 30/60] 10.222.0.16 () {64 vars in 1235 bytes} [Sun Oct 30 00:49:44 2022] POST /compute/v2.1/flavors => generated 409 bytes in 432 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 30 00:49:46.081226 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi [None req-0549f73d-759d-4f38-9280-068363435135 admin admin] Calling method '>' {{(pid=103760) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:49:46.082233 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: INFO nova.api.openstack.requestlog [None req-0549f73d-759d-4f38-9280-068363435135 admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001447 Oct 30 00:49:46.082635 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: [pid: 103760|app: 0|req: 31/61] 10.222.0.16 () {56 vars in 1074 bytes} [Sun Oct 30 00:49:46 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:49:46.624299 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi [None req-6bd2dae5-95a0-406f-8852-b386fc02fbed admin admin] Calling method '>' {{(pid=103759) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:49:46.644741 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: INFO nova.api.openstack.requestlog [None req-6bd2dae5-95a0-406f-8852-b386fc02fbed admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 5777 microversion: 2.92 time: 0.427147 Oct 30 00:49:46.645271 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: [pid: 103759|app: 0|req: 31/62] 10.222.0.16 () {60 vars in 1202 bytes} [Sun Oct 30 00:49:46 2022] GET /compute/v2.1/flavors/detail => generated 5777 bytes in 429 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 00:50:44.269689 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi [None req-0549f73d-759d-4f38-9280-068363435135 admin admin] Calling method '>' {{(pid=103760) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:50:44.270730 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: INFO nova.api.openstack.requestlog [None req-0549f73d-759d-4f38-9280-068363435135 admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001508 Oct 30 00:50:44.271136 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: [pid: 103760|app: 0|req: 32/63] 10.222.0.16 () {54 vars in 931 bytes} [Sun Oct 30 00:50:44 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:50:44.282389 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG dbcounter [-] [103759] Writing DB stats nova_api:INSERT=5,nova_api:SELECT=28 {{(pid=103759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:44.716408 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi [None req-2fc6c077-f8fa-4c01-86ec-c4bf2cf0b711 admin admin] Calling method '>' {{(pid=103759) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:50:44.740957 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: INFO nova.api.openstack.requestlog [None req-2fc6c077-f8fa-4c01-86ec-c4bf2cf0b711 admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 5791 microversion: 2.61 time: 0.463143 Oct 30 00:50:44.741927 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: [pid: 103759|app: 0|req: 32/64] 10.222.0.16 () {60 vars in 1292 bytes} [Sun Oct 30 00:50:44 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 5791 bytes in 464 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 00:50:44.763419 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG dbcounter [-] [103760] Writing DB stats nova_api:SELECT=14 {{(pid=103760) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:44.767121 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi [None req-f2cf78d6-8dca-421e-922f-bdd989f4cfea admin admin] Calling method '>' {{(pid=103760) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:50:44.774905 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: INFO nova.api.openstack.requestlog [None req-f2cf78d6-8dca-421e-922f-bdd989f4cfea admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/42/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.012769 Oct 30 00:50:44.775416 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: [pid: 103760|app: 0|req: 33/65] 10.222.0.16 () {60 vars in 1283 bytes} [Sun Oct 30 00:50:44 2022] GET /compute/v2.1/flavors/42/os-extra_specs => generated 19 bytes in 14 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:50:44.787924 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi [None req-0122cf22-3634-4ba2-ad7e-e02a62d75e9b admin admin] Calling method '>' {{(pid=103759) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:50:44.795828 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: INFO nova.api.openstack.requestlog [None req-0122cf22-3634-4ba2-ad7e-e02a62d75e9b admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/451/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011724 Oct 30 00:50:44.796331 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: [pid: 103759|app: 0|req: 33/66] 10.222.0.16 () {60 vars in 1286 bytes} [Sun Oct 30 00:50:44 2022] GET /compute/v2.1/flavors/451/os-extra_specs => generated 19 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 00:50:44.808335 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi [None req-48fd4621-639c-4048-b835-83b6c9eff5ba admin admin] Calling method '>' {{(pid=103760) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:50:44.815688 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: INFO nova.api.openstack.requestlog [None req-48fd4621-639c-4048-b835-83b6c9eff5ba admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/452/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011160 Oct 30 00:50:44.816271 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: [pid: 103760|app: 0|req: 34/67] 10.222.0.16 () {60 vars in 1286 bytes} [Sun Oct 30 00:50:44 2022] GET /compute/v2.1/flavors/452/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:50:44.831261 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi [None req-e1aebcc4-690c-43c1-af05-982cc8f71805 admin admin] Calling method '>' {{(pid=103759) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:50:44.841826 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: INFO nova.api.openstack.requestlog [None req-e1aebcc4-690c-43c1-af05-982cc8f71805 admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/84/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.016691 Oct 30 00:50:44.842468 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: [pid: 103759|app: 0|req: 34/68] 10.222.0.16 () {60 vars in 1283 bytes} [Sun Oct 30 00:50:44 2022] GET /compute/v2.1/flavors/84/os-extra_specs => generated 19 bytes in 18 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:53:50.389886 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi [None req-48fd4621-639c-4048-b835-83b6c9eff5ba admin admin] Calling method '>' {{(pid=103760) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:53:50.391083 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: INFO nova.api.openstack.requestlog [None req-48fd4621-639c-4048-b835-83b6c9eff5ba admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001740 Oct 30 00:53:50.391516 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: [pid: 103760|app: 0|req: 35/69] 10.222.0.16 () {56 vars in 1074 bytes} [Sun Oct 30 00:53:50 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:53:50.533537 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG dbcounter [-] [103759] Writing DB stats nova_api:SELECT=6 {{(pid=103759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:53:50.702831 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi [None req-ff7bec5d-85bc-42c1-9d50-5fe9fec30eef admin admin] Calling method '>' {{(pid=103759) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:53:50.707859 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG oslo_concurrency.lockutils [None req-ff7bec5d-85bc-42c1-9d50-5fe9fec30eef admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103759) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:53:50.708340 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG oslo_concurrency.lockutils [None req-ff7bec5d-85bc-42c1-9d50-5fe9fec30eef admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=103759) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:53:50.708773 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG oslo_concurrency.lockutils [None req-ff7bec5d-85bc-42c1-9d50-5fe9fec30eef admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103759) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:53:50.709492 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG oslo_concurrency.lockutils [None req-ff7bec5d-85bc-42c1-9d50-5fe9fec30eef admin admin] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103759) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:53:50.709940 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG oslo_concurrency.lockutils [None req-ff7bec5d-85bc-42c1-9d50-5fe9fec30eef admin admin] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103759) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:53:50.710365 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG oslo_concurrency.lockutils [None req-ff7bec5d-85bc-42c1-9d50-5fe9fec30eef admin admin] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103759) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:53:50.741896 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: INFO nova.api.openstack.requestlog [None req-ff7bec5d-85bc-42c1-9d50-5fe9fec30eef admin admin] 10.222.0.16 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.92 time: 0.211090 Oct 30 00:53:50.742464 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: [pid: 103759|app: 0|req: 35/70] 10.222.0.16 () {60 vars in 1193 bytes} [Sun Oct 30 00:53:50 2022] GET /compute/v2.1/os-services => generated 791 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 30 00:53:51.809561 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi [None req-48fd4621-639c-4048-b835-83b6c9eff5ba admin admin] Calling method '>' {{(pid=103760) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:53:51.810563 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: INFO nova.api.openstack.requestlog [None req-48fd4621-639c-4048-b835-83b6c9eff5ba admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001457 Oct 30 00:53:51.810975 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: [pid: 103760|app: 0|req: 36/71] 10.222.0.16 () {56 vars in 1074 bytes} [Sun Oct 30 00:53:51 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:53:52.015441 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi [None req-5009aa12-08dd-418a-8372-6d48ebd91dd7 admin admin] Calling method '>' {{(pid=103759) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:53:52.021945 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG oslo_concurrency.lockutils [None req-5009aa12-08dd-418a-8372-6d48ebd91dd7 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103759) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:53:52.022445 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG oslo_concurrency.lockutils [None req-5009aa12-08dd-418a-8372-6d48ebd91dd7 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=103759) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:53:52.022867 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG oslo_concurrency.lockutils [None req-5009aa12-08dd-418a-8372-6d48ebd91dd7 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103759) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:53:52.023573 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG oslo_concurrency.lockutils [None req-5009aa12-08dd-418a-8372-6d48ebd91dd7 admin admin] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103759) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:53:52.024040 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG oslo_concurrency.lockutils [None req-5009aa12-08dd-418a-8372-6d48ebd91dd7 admin admin] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103759) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:53:52.024446 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG oslo_concurrency.lockutils [None req-5009aa12-08dd-418a-8372-6d48ebd91dd7 admin admin] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103759) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:53:52.044835 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: INFO nova.api.openstack.requestlog [None req-5009aa12-08dd-418a-8372-6d48ebd91dd7 admin admin] 10.222.0.16 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.92 time: 0.097800 Oct 30 00:53:52.045626 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: [pid: 103759|app: 0|req: 36/72] 10.222.0.16 () {60 vars in 1193 bytes} [Sun Oct 30 00:53:51 2022] GET /compute/v2.1/os-services => generated 791 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 30 00:54:03.557271 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi [None req-48fd4621-639c-4048-b835-83b6c9eff5ba admin admin] Calling method '>' {{(pid=103760) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:54:03.558272 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: INFO nova.api.openstack.requestlog [None req-48fd4621-639c-4048-b835-83b6c9eff5ba admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001431 Oct 30 00:54:03.558637 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: [pid: 103760|app: 0|req: 37/73] 10.222.0.16 () {56 vars in 1074 bytes} [Sun Oct 30 00:54:03 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:54:03.706764 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG dbcounter [-] [103759] Writing DB stats nova_cell0:SELECT=4 {{(pid=103759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:54:03.707450 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG dbcounter [-] [103759] Writing DB stats nova_cell1:SELECT=4 {{(pid=103759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:54:03.708327 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG dbcounter [-] [103759] Writing DB stats nova_api:SELECT=8 {{(pid=103759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:54:03.764396 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi [None req-cddeecbc-5413-4116-aef2-ac3978c543e9 admin admin] Calling method '>' {{(pid=103759) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:54:03.769824 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG oslo_concurrency.lockutils [None req-cddeecbc-5413-4116-aef2-ac3978c543e9 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103759) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:54:03.770488 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG oslo_concurrency.lockutils [None req-cddeecbc-5413-4116-aef2-ac3978c543e9 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=103759) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:54:03.771125 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG oslo_concurrency.lockutils [None req-cddeecbc-5413-4116-aef2-ac3978c543e9 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=103759) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:54:03.772234 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG oslo_concurrency.lockutils [None req-cddeecbc-5413-4116-aef2-ac3978c543e9 admin admin] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103759) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:54:03.772859 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG oslo_concurrency.lockutils [None req-cddeecbc-5413-4116-aef2-ac3978c543e9 admin admin] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=103759) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:54:03.773449 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG oslo_concurrency.lockutils [None req-cddeecbc-5413-4116-aef2-ac3978c543e9 admin admin] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=103759) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:54:03.800763 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: INFO nova.api.openstack.requestlog [None req-cddeecbc-5413-4116-aef2-ac3978c543e9 admin admin] 10.222.0.16 "GET /compute/v2.1/os-services" status: 200 len: 1011 microversion: 2.92 time: 0.101674 Oct 30 00:54:03.801315 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: [pid: 103759|app: 0|req: 37/74] 10.222.0.16 () {60 vars in 1193 bytes} [Sun Oct 30 00:54:03 2022] GET /compute/v2.1/os-services => generated 1011 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 00:54:04.873418 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG nova.api.openstack.wsgi [None req-48fd4621-639c-4048-b835-83b6c9eff5ba admin admin] Calling method '>' {{(pid=103760) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:54:04.874431 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: INFO nova.api.openstack.requestlog [None req-48fd4621-639c-4048-b835-83b6c9eff5ba admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001467 Oct 30 00:54:04.874829 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: [pid: 103760|app: 0|req: 38/75] 10.222.0.16 () {56 vars in 1074 bytes} [Sun Oct 30 00:54:04 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:54:05.074442 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG nova.api.openstack.wsgi [None req-47edc1ba-967e-47fb-a18a-49b7043f9544 admin admin] Calling method '>' {{(pid=103759) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:54:05.078440 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG oslo_concurrency.lockutils [None req-47edc1ba-967e-47fb-a18a-49b7043f9544 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103759) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:54:05.078776 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG oslo_concurrency.lockutils [None req-47edc1ba-967e-47fb-a18a-49b7043f9544 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=103759) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:54:05.079213 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG oslo_concurrency.lockutils [None req-47edc1ba-967e-47fb-a18a-49b7043f9544 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103759) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:54:05.079991 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG oslo_concurrency.lockutils [None req-47edc1ba-967e-47fb-a18a-49b7043f9544 admin admin] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103759) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:54:05.080440 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG oslo_concurrency.lockutils [None req-47edc1ba-967e-47fb-a18a-49b7043f9544 admin admin] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103759) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:54:05.080857 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: DEBUG oslo_concurrency.lockutils [None req-47edc1ba-967e-47fb-a18a-49b7043f9544 admin admin] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103759) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:54:05.102888 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: INFO nova.api.openstack.requestlog [None req-47edc1ba-967e-47fb-a18a-49b7043f9544 admin admin] 10.222.0.16 "GET /compute/v2.1/os-services" status: 200 len: 1011 microversion: 2.92 time: 0.088197 Oct 30 00:54:05.103465 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: [pid: 103759|app: 0|req: 38/76] 10.222.0.16 () {60 vars in 1193 bytes} [Sun Oct 30 00:54:05 2022] GET /compute/v2.1/os-services => generated 1011 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 00:55:03.384744 dv-cinder-master-3-iscsi devstack@n-api.service[103758]: Sun Oct 30 00:55:03 2022 - graceful shutdown triggered... Oct 30 00:55:03.385299 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: Gracefully killing worker 2 (pid: 103760)... Oct 30 00:55:03.384817 dv-cinder-master-3-iscsi systemd[1]: Stopping Devstack devstack@n-api.service... Oct 30 00:55:03.385990 dv-cinder-master-3-iscsi devstack@n-api.service[103759]: Gracefully killing worker 1 (pid: 103759)... Oct 30 00:55:03.528584 dv-cinder-master-3-iscsi devstack@n-api.service[103760]: DEBUG dbcounter [-] [103760] Writing DB stats nova_api:SELECT=4 {{(pid=103760) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:55:04.385918 dv-cinder-master-3-iscsi devstack@n-api.service[103758]: worker 1 buried after 1 seconds Oct 30 00:55:04.385918 dv-cinder-master-3-iscsi devstack@n-api.service[103758]: worker 2 buried after 1 seconds Oct 30 00:55:04.385918 dv-cinder-master-3-iscsi devstack@n-api.service[103758]: goodbye to uWSGI. Oct 30 00:55:04.387655 dv-cinder-master-3-iscsi systemd[1]: devstack@n-api.service: Succeeded. Oct 30 00:55:04.388279 dv-cinder-master-3-iscsi systemd[1]: Stopped Devstack devstack@n-api.service. Oct 30 00:55:04.391199 dv-cinder-master-3-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Oct 30 00:55:04.401936 dv-cinder-master-3-iscsi devstack@n-api.service[118958]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Oct 30 00:55:04.404169 dv-cinder-master-3-iscsi devstack@n-api.service[118958]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sun Oct 30 00:55:04 2022] *** Oct 30 00:55:04.404169 dv-cinder-master-3-iscsi devstack@n-api.service[118958]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 30 00:55:04.404169 dv-cinder-master-3-iscsi devstack@n-api.service[118958]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 30 00:55:04.404169 dv-cinder-master-3-iscsi devstack@n-api.service[118958]: nodename: dv-cinder-master-3-iscsi Oct 30 00:55:04.404169 dv-cinder-master-3-iscsi devstack@n-api.service[118958]: machine: x86_64 Oct 30 00:55:04.404169 dv-cinder-master-3-iscsi devstack@n-api.service[118958]: clock source: unix Oct 30 00:55:04.404169 dv-cinder-master-3-iscsi devstack@n-api.service[118958]: pcre jit disabled Oct 30 00:55:04.404169 dv-cinder-master-3-iscsi devstack@n-api.service[118958]: detected number of CPU cores: 4 Oct 30 00:55:04.404169 dv-cinder-master-3-iscsi devstack@n-api.service[118958]: current working directory: / Oct 30 00:55:04.404169 dv-cinder-master-3-iscsi devstack@n-api.service[118958]: detected binary path: /usr/bin/uwsgi-core Oct 30 00:55:04.404169 dv-cinder-master-3-iscsi devstack@n-api.service[118958]: your processes number limit is 47680 Oct 30 00:55:04.404169 dv-cinder-master-3-iscsi devstack@n-api.service[118958]: your memory page size is 4096 bytes Oct 30 00:55:04.404169 dv-cinder-master-3-iscsi devstack@n-api.service[118958]: detected max file descriptor number: 2048 Oct 30 00:55:04.404169 dv-cinder-master-3-iscsi devstack@n-api.service[118958]: lock engine: pthread robust mutexes Oct 30 00:55:04.405152 dv-cinder-master-3-iscsi devstack@n-api.service[118958]: thunder lock: enabled Oct 30 00:55:04.405152 dv-cinder-master-3-iscsi devstack@n-api.service[118958]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Oct 30 00:55:04.405152 dv-cinder-master-3-iscsi devstack@n-api.service[118958]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 30 00:55:04.448313 dv-cinder-master-3-iscsi devstack@n-api.service[118958]: Python main interpreter initialized at 0x5633d75709c0 Oct 30 00:55:04.448313 dv-cinder-master-3-iscsi devstack@n-api.service[118958]: python threads support enabled Oct 30 00:55:04.448313 dv-cinder-master-3-iscsi devstack@n-api.service[118958]: your server socket listen backlog is limited to 100 connections Oct 30 00:55:04.448313 dv-cinder-master-3-iscsi devstack@n-api.service[118958]: your mercy for graceful operations on workers is 90 seconds Oct 30 00:55:04.448776 dv-cinder-master-3-iscsi devstack@n-api.service[118958]: mapped 403077 bytes (393 KB) for 2 cores Oct 30 00:55:04.448776 dv-cinder-master-3-iscsi devstack@n-api.service[118958]: *** Operational MODE: preforking *** Oct 30 00:55:04.449063 dv-cinder-master-3-iscsi devstack@n-api.service[118958]: *** uWSGI is running in multiple interpreter mode *** Oct 30 00:55:04.449063 dv-cinder-master-3-iscsi devstack@n-api.service[118958]: spawned uWSGI master process (pid: 118958) Oct 30 00:55:04.449282 dv-cinder-master-3-iscsi systemd[1]: Started Devstack devstack@n-api.service. Oct 30 00:55:04.449837 dv-cinder-master-3-iscsi devstack@n-api.service[118958]: spawned uWSGI worker 1 (pid: 118961, cores: 1) Oct 30 00:55:04.450405 dv-cinder-master-3-iscsi devstack@n-api.service[118958]: spawned uWSGI worker 2 (pid: 118962, cores: 1) Oct 30 00:55:04.450737 dv-cinder-master-3-iscsi devstack@n-api.service[118958]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 30 00:55:05.099007 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Oct 30 00:55:05.104979 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Oct 30 00:55:06.323124 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 30 00:55:06.323124 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: from cryptography.utils import int_from_bytes Oct 30 00:55:06.323993 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 30 00:55:06.323993 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: from cryptography.utils import int_from_bytes Oct 30 00:55:06.358281 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 30 00:55:06.358281 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: from cryptography.utils import int_from_bytes Oct 30 00:55:06.358811 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 30 00:55:06.358811 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: from cryptography.utils import int_from_bytes Oct 30 00:55:08.783952 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 30 00:55:08.784608 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 30 00:55:08.784721 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 30 00:55:08.785084 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 30 00:55:08.785362 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 30 00:55:08.785741 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.786092 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.786413 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.786737 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.787075 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.787390 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.787843 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.788291 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.788551 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 30 00:55:08.788928 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.789015 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 30 00:55:08.789120 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.789314 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.789417 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 30 00:55:08.789616 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.789733 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 30 00:55:08.789956 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.790055 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 30 00:55:08.790271 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.790377 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.790592 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.790704 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.790931 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.791029 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.791228 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.791334 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.791559 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.791669 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.791901 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.792016 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.792260 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.792438 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.792556 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.794013 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.794144 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.794144 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.794303 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.794384 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.794469 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.794547 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.794629 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.794700 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.794783 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.794866 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.794943 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.795020 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.795096 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.795192 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.795271 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.795349 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.795420 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.795842 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.796068 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.796319 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.796536 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.796536 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 4.0 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.796932 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.797165 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.797377 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.797600 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.0 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.797802 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.797802 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.798204 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.798414 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.798623 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.798854 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.799055 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.799263 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.799475 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.799692 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.799979 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.800201 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.800413 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.800591 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.800809 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.800976 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.801155 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.801312 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.801490 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.801633 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 4.0 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.801779 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.801929 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.802091 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.802264 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.0 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.802435 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.802435 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.802707 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.802859 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.803042 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.803211 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.803578 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.803848 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.804018 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.805367 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.805531 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.805691 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.805838 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.805976 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.806163 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.806347 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.806499 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.806639 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.806639 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.806913 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.807064 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.807213 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.807368 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.807517 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.807667 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.807877 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.808030 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.808177 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.808316 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.808462 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.808608 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.808756 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.808895 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.809039 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.809184 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.809334 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.809479 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.809631 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.809778 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.809916 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.810053 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.810199 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.810348 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.810498 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.810640 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.810788 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.810932 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.811086 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.811240 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.811390 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.811530 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.811675 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.811675 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.811978 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.16 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.811978 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.16 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.812278 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.812920 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.813077 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.813077 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.813365 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.813605 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.813761 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.814013 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.814159 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.814382 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.814528 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.814676 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.814835 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.815335 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.815489 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.815636 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.815829 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.816115 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.816271 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.816425 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.816566 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.816704 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.816855 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.816855 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.817136 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.16 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.817280 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.817425 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.16 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.817570 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.817722 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.817871 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.818017 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.818162 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.818307 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.818914 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.819112 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.819270 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.819420 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.819697 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.819869 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.820028 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.820169 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.820306 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.820455 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.820594 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.820741 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.820890 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.821029 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.821173 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.821320 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.821458 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.821602 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.821749 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.821898 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.822037 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.822174 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.822329 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.822480 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.822631 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.822770 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.822909 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.823052 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.823201 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.823353 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.823498 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.823498 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.823914 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.824174 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.824323 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.824568 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.824714 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.824865 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.825022 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.825516 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.825671 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.825811 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.825949 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.826105 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.826337 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.826489 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.826635 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.826782 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.826930 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.827075 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.827227 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.827365 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.827518 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.827663 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.827838 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.827988 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.828721 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.828875 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.829025 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.829178 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.829331 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.829471 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.829608 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.829747 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.830134 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.830293 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.830432 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.830580 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.830718 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.830864 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.831008 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.831165 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.831305 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.831452 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.831597 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.831757 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.831903 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.832052 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.832233 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.832387 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.832534 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.832672 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.832809 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.832955 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.833118 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.833258 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.833397 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.833544 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.833683 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.833930 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.834205 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.834361 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.834579 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.834886 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.835050 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.835231 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.835960 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.836125 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.836268 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.836526 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.836683 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.836683 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.836964 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.837111 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.837257 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.837397 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.837543 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.837693 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.837967 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.838135 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.838285 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.838430 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.838580 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.838727 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.838865 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.839003 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.839140 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.839288 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.839435 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.839586 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.839749 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.839903 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:55:08.840061 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.840207 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.840359 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.840508 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.840654 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.840800 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.840945 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.841083 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.841230 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.841230 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.841506 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.841647 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.841784 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.842167 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.842334 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.842487 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.842626 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.842774 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.842920 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.843058 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.843188 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.843507 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.843663 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.843841 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.844082 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.844311 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.844451 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.844553 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.844640 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.844730 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.844978 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.845172 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.845286 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.845373 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.845483 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.845738 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.845954 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.846047 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.846141 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.846227 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.846310 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.846401 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.846513 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.846652 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.846750 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.846837 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.846927 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.847011 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.847094 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.847190 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.847284 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.847370 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.847370 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.847528 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.847612 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.847703 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.847844 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.847931 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.847931 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.848091 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.848175 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.848312 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.848402 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.848486 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.848570 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.848661 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.848661 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.848824 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.848908 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.848991 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.849073 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.849163 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.849246 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.849337 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.849428 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.849513 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.849602 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.849693 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.849784 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.849876 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.849960 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.850068 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.850196 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.850289 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.850381 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.850466 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.850549 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.850631 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.850763 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.850851 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.851011 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.851102 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.851192 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.851297 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.851380 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.851462 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.851554 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.851639 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.851738 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.851838 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.851930 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.852015 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.852110 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.852196 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.852279 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.852363 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.852451 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.852548 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.852640 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.852725 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.852815 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.852898 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.852981 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.852981 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.853154 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.853240 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.853322 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.853405 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.853487 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.853581 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.853665 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.853756 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.853840 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.853930 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.854019 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.854180 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.854299 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.854393 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.854477 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.854559 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.854651 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.854737 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.854821 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.854907 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.855010 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.855103 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.855199 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.855284 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.855378 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.855378 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.855537 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.855537 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.855960 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.855960 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.856216 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.856305 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.856389 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.856479 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.856573 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.856691 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.856825 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.857001 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.857099 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.857310 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.857409 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.857618 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.857717 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.857926 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.858026 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.858238 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.858337 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.858543 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.858641 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.858850 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.858948 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.859157 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.859253 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.859463 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.859561 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.859792 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.859900 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.860119 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.860218 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.860428 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.860527 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.860734 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.860833 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.861038 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.861136 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.861344 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.861434 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.861663 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.861768 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.861977 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.862075 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.862288 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.862378 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.862597 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.862701 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.862905 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.862996 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.863214 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.863312 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.863519 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.863618 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.863850 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.863955 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.864174 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.864271 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.864484 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.864607 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.864791 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.864907 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.865180 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.865304 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.865424 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.865532 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.865919 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.866057 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.866269 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.866413 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.866597 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.866745 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.866964 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.867100 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.867212 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.867386 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.867532 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.867869 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.867992 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.868115 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.868233 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.868342 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.868525 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.868685 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.868992 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.869107 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.869316 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.869445 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.869632 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.869756 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.870012 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.870137 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.870246 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.870367 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.870556 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.870680 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.870910 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.871023 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.871181 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.871270 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.871494 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.871616 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.871843 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.871941 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.872173 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.872279 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.872491 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.872590 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.872817 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.872914 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.873127 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.873227 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.873438 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.873528 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.873761 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.873861 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.874069 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.874161 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.874387 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.874513 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.874719 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.874808 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.875033 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.875124 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.875353 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.875455 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.875659 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.875809 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.876010 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.876145 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.876324 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.876424 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.876631 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.876729 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.876943 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.877056 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.877251 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.877356 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.877558 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.877665 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.877870 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.877966 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.878183 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.878286 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.878495 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.878606 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.878807 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.878903 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.879111 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.879206 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.879427 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.879532 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.879758 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.879857 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.880071 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.880169 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.880379 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.880484 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.880691 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.880794 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.881013 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.881109 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.881373 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.881481 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.881729 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.881915 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.882229 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.882332 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.882544 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.882646 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.882854 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.882953 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.883182 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.883272 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.883495 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.883590 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.883839 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.883933 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.884150 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.884242 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.884459 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.884584 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.884782 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.884882 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.885097 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.885386 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.885386 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.885551 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.885703 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.885794 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.886027 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.886154 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.886338 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.886463 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.886653 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.886753 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.886966 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.887065 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.887281 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.887392 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.887587 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.887692 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.887922 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.888025 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.888236 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.888333 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.888547 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.888637 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.888856 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.888948 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.889164 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.889254 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.889471 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.889560 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.889791 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.889881 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.890104 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.890195 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.890407 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.890496 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.890719 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.890818 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.891027 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.891119 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.891338 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.891436 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.891655 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.891782 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.892009 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.892276 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.892368 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.892602 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.892701 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.892921 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.893022 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.893233 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.893331 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.893547 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.893646 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.893860 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.893966 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.894174 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.894273 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.894489 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.894579 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.894812 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.894911 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.895125 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.895216 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.895439 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.895576 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.895795 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.895909 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.896078 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.896170 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.896395 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.896509 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.896729 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.896837 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.897087 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.897515 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.897623 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.897837 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.897934 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.898146 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.898243 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.898456 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.898570 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.898766 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.898922 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.899074 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.899251 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.899395 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.899593 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.899692 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.899961 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.900066 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.900288 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.900379 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.900602 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.900693 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.900921 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.901019 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.901228 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.901328 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.901534 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.901631 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.901837 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.902105 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.902204 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.902297 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.902444 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.902537 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.902751 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.902842 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.903068 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.903169 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.903376 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.903476 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.903682 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.903841 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.904012 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.904105 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.904320 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.904420 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.904641 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.904787 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.904951 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.905045 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.905266 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.905357 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.905576 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.905715 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.905887 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.905987 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.906196 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.906322 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.906508 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.906613 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.906820 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.906973 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.907126 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.907232 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.907444 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.907568 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.907762 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.907870 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.908080 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.908179 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.908391 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.908492 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.908709 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.908808 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.909023 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.909122 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.909330 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.909433 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.909635 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.909733 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.909947 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.910038 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.910258 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.910357 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.910564 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.910655 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.910877 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.910975 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.911182 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.911273 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.911486 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.911576 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.911817 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.911920 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.912152 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.912258 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.912427 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.912552 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.912952 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.913046 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.913492 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.913585 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.913809 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.913902 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.914129 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.914227 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.914442 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.914533 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.914753 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.914844 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.915064 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.915233 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.915375 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.915466 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.915683 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.915835 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.916015 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.916120 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.916326 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.916463 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.916654 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.916757 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.916972 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.917186 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.917351 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.917439 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.917582 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.917724 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.917890 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.917993 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.918195 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.918292 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.918498 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.918588 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.918809 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.918907 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.919110 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.919200 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.919417 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.919522 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.919755 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.919853 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.920052 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.920143 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.920361 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.920466 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.920673 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.920764 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.920979 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.921078 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.921284 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.921382 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.921592 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.921691 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.921896 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.921988 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.922227 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.922352 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.922539 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.922644 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.922850 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.922953 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.923162 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.923255 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.923472 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.923571 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.923801 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.923909 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.924113 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.924205 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.924420 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.924519 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.924728 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.924826 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.925032 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.925123 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.925337 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.925428 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.925643 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.925742 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.925955 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.926053 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.926263 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.926354 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.926567 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.926657 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.926875 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.926969 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.927179 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.927270 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.927489 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.927785 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.927880 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.927966 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.928233 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.928328 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.928727 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.928828 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.929041 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.929177 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.929358 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.929513 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.929672 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.929838 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.929985 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.930159 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.930300 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.930481 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.930612 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.930792 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.930929 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.931118 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.931244 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.931437 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.931558 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.931775 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.931908 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.932089 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.932223 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.932454 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.932550 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.932711 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.932853 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.933025 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.933139 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.933342 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.933443 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.933668 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.933766 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.933994 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.934094 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.934321 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.934426 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.934627 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.934725 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.934933 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.935026 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.935245 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.935348 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.935549 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.935647 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.935885 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.936205 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.936306 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.936521 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.936648 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.936836 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.936935 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.937150 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.937256 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.937463 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.937556 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.937778 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.937867 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.938104 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.938203 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.938385 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.938492 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.938696 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.938787 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.939005 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.939120 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.939331 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.939434 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.939642 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.939784 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.939978 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.940105 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.940289 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.940415 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.940596 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.940726 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.940912 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.941039 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.941219 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.941367 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.941674 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.941910 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.942014 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.942233 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.942325 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.942555 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.942643 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.942853 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.942947 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.943171 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.943262 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.943862 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.944133 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.944381 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.944657 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.944812 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.944981 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.945133 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.945407 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.945543 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.945728 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.945864 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.946048 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.946190 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.946376 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.946509 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.946704 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.946804 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.947036 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.947134 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.947348 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.947483 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.947667 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.947784 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.948009 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.948135 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.948339 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.948431 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.948642 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.948733 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.948970 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.949081 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.949286 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.949405 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.949596 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.949691 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.949924 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.950024 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.950253 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.950351 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.950572 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.950679 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.950899 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.951006 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.951311 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.951407 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.951624 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.951758 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.951968 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.952065 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.952289 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.952392 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.952617 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.952728 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.952923 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.953031 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.953232 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.953336 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.953538 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.953637 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.953848 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.953994 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.954159 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.954265 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.954472 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.954572 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.954778 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.954877 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.955087 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.955192 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.955398 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.955498 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.955709 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.955816 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.956042 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.956142 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.956351 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.956444 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.956676 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.956767 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.956985 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.957108 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.957313 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.957413 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.957630 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.957729 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.957954 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.958053 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.958278 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.958377 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.958585 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.958676 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.958899 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.959024 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.959208 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.959351 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.959640 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.959859 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.960099 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.960214 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.960565 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.960656 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.960887 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.960987 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.961209 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.961310 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.961561 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.961671 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.961872 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.961964 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.962182 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.962283 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.962514 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.962629 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.962828 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.962940 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.963154 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.963260 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.963474 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.963566 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.963823 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.963920 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.964152 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.964251 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.964461 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.964561 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.964781 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.964918 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.965096 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.965189 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.965402 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.965510 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.965710 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.965801 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.966023 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.966126 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.966332 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.966431 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.966648 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.966748 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.966956 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.967056 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.967264 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.967362 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.967572 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.967672 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.967925 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.968022 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.968236 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.968335 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.968564 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.968673 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.968878 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.968977 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.969190 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.969280 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.969499 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.969613 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.969811 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.969914 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.970124 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.970214 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.970433 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.970534 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.970745 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.970838 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.971054 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.971158 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.971365 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.971458 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.971674 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.971838 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.972009 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.972109 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.972315 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.972408 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.972624 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.972724 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.972931 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.973026 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.973257 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.973399 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.973564 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.973656 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.973878 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.973977 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.974187 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.974278 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.974494 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.974592 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.974802 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.974902 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.975143 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.975392 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.975741 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.975837 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.976073 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.976185 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.976393 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.976485 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.976713 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.976815 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.977032 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.977133 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.977352 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.977451 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.977671 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.977774 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.977990 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.978096 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.978307 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.978407 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.978622 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.978718 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.978944 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.979044 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.979268 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.979366 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.979582 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.979680 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.979930 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.980025 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.980255 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.980350 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.980572 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.980670 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.980892 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.981104 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.981196 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.981437 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.981551 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.981759 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.981888 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.982079 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.982197 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.982394 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.982486 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.982712 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.982804 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.983030 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.983124 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.983344 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.983443 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.983661 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.983790 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.984001 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.984102 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.984314 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.984414 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.984625 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.984717 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.984940 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.985036 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.985258 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.985358 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.985576 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.985670 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.985901 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.985994 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.986217 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.986309 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.986537 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.986630 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.986855 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.986954 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.987172 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.987273 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.987492 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.987596 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.987836 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.988043 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.988152 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.988366 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.988465 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.988703 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.988795 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.989033 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.989126 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.989360 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.989459 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.989675 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.989772 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.989990 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.990090 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.990303 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.992671 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.992786 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.992877 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.992877 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.993103 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.993197 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.993290 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.993384 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.993489 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.993576 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.993667 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.993754 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.993754 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.993909 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.994011 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.994104 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.994197 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.994282 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.994414 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.994513 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.994615 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.994701 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.994701 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.994862 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.994954 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.995040 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.995040 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.995210 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.995304 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.995389 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.995621 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.995849 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.995964 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.996065 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.996293 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.996386 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.996604 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.996704 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.996912 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.997004 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.997224 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.997322 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.997536 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.997638 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.997841 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.997933 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.998142 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.998233 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.998455 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.998559 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.998762 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.998854 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.999063 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.999154 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.999374 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.999466 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.999680 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:08.999941 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.000110 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.000301 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.000405 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.000624 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.000716 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.000954 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.001053 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.001281 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.001381 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.001601 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.001693 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.001908 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.002001 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.002220 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.002319 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.002526 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.002618 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.002834 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.002932 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.003138 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.003276 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.003449 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.003540 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.003771 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.003913 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.004171 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.004300 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.004410 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.004528 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.004685 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.004778 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.004994 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.005087 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.005304 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.005397 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.005619 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.005714 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.005923 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.006015 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.006258 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.006409 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.006595 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.006698 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.006921 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.007187 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.007280 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.007516 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.007611 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.007858 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.007957 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.008176 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.008278 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.008490 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.008737 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.008838 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.009083 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.009177 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.009405 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.009497 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.009720 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.009820 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.010032 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.010124 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.010350 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.010450 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.010658 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.010750 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.010969 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.011061 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.011296 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.011396 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.011609 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.011710 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.011946 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.012039 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.012255 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.012355 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.012567 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.012666 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.012878 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.012970 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.013192 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.013326 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.013502 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.013595 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.013813 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.013905 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.014123 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.014222 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.014434 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.014526 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.014747 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.014987 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.015087 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.015326 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.015418 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.015656 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.015811 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.016002 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.016102 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.016336 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.016435 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.016646 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.016739 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.016956 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.017098 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.017264 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.017390 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.017582 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.017719 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.017891 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.017993 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.018202 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.018292 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.018514 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.018643 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.018823 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.018923 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.019130 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.019220 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.019442 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.019541 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.019771 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.019879 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.020083 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.020175 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.020394 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.020506 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.020709 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.020824 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.021022 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.021145 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.021336 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.021473 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.021647 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.021786 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.022057 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.022262 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.022528 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.022631 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.022901 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.023045 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.023222 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.023379 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.023536 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.023707 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.023906 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.024054 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.024250 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.024381 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.024585 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.024685 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.024913 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.025006 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.025223 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.025315 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.025532 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.025698 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.025839 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.025939 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.026152 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.026251 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.026472 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.026565 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.026785 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.026884 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.027092 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.027257 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.027396 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.027584 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.027704 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.027937 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.028045 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.028264 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.028367 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.028583 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.028676 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.028910 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.029010 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.029234 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.029327 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.029522 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.029614 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.029829 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.029928 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.030132 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.030225 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.030438 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.030539 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.030743 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.030841 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.031050 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.031150 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.031360 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.031451 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.031667 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.031791 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.032035 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.032133 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.032345 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.032437 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.032659 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.032764 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.032962 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.033060 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.033278 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.033369 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.033587 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.033688 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.033897 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.033989 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.034209 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.034309 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.034516 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.034616 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.034825 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.034924 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.035133 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.035225 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.035445 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.035538 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.035777 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.035885 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.036085 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.036187 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.036392 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.036485 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.036700 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.036794 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.037006 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.037151 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.037312 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.037405 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.037641 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.037746 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.037975 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.038200 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.038318 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.038550 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.038645 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.038869 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.038963 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.039182 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.039276 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.039495 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.039595 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.039830 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.039937 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.040157 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.040251 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.040481 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.040621 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.040802 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.040902 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.041117 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.041216 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.041431 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.041522 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.041753 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.041852 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.042065 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.042156 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.042375 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.042472 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.042689 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.042789 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.043003 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.043097 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.043317 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.043408 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.043633 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.043782 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.043991 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.044090 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.044309 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.044410 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.044625 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.044725 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.044941 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.045136 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.045256 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.045349 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.045568 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.045660 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.045897 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.045989 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.046224 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.046316 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.046529 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.046627 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.046858 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.046950 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.047181 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.047276 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.047506 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.047604 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.047848 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.047944 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.048164 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.048280 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.048472 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.048566 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.048722 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=118961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 30 00:55:09.048814 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.049094 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.049406 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.049722 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.050035 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.050356 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.050668 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.050983 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.051303 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.051623 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.051976 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.052307 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.052620 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.052952 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.053390 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.053923 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.054249 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.054603 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.054943 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.055265 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.055587 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.055932 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:55:09.056185 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=118962) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 30 00:55:09.112701 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO dbcounter [None req-3a311086-d895-4715-8848-7f0ede97d571 None None] Registered counter for database nova_api Oct 30 00:55:09.120234 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO dbcounter [None req-b80acdf6-3a3a-43eb-8479-b2cad9a13728 None None] Registered counter for database nova_api Oct 30 00:55:09.133449 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_db.sqlalchemy.engines [None req-3a311086-d895-4715-8848-7f0ede97d571 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=118961) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 30 00:55:09.140300 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writer thread running {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 30 00:55:09.140472 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_db.sqlalchemy.engines [None req-b80acdf6-3a3a-43eb-8479-b2cad9a13728 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=118962) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 30 00:55:09.147113 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writer thread running {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 30 00:55:09.197797 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.context [None req-3a311086-d895-4715-8848-7f0ede97d571 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),a2b915ba-4703-4df7-87fc-723ef53ef868(cell1) {{(pid=118961) load_cells /opt/stack/nova/nova/context.py:464}} Oct 30 00:55:09.201874 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3a311086-d895-4715-8848-7f0ede97d571 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:55:09.202318 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3a311086-d895-4715-8848-7f0ede97d571 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:55:09.203974 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3a311086-d895-4715-8848-7f0ede97d571 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:55:09.204720 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3a311086-d895-4715-8848-7f0ede97d571 None None] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:55:09.204962 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.context [None req-b80acdf6-3a3a-43eb-8479-b2cad9a13728 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),a2b915ba-4703-4df7-87fc-723ef53ef868(cell1) {{(pid=118962) load_cells /opt/stack/nova/nova/context.py:464}} Oct 30 00:55:09.205148 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3a311086-d895-4715-8848-7f0ede97d571 None None] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:55:09.207186 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3a311086-d895-4715-8848-7f0ede97d571 None None] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:55:09.209001 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b80acdf6-3a3a-43eb-8479-b2cad9a13728 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:55:09.209225 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO dbcounter [None req-3a311086-d895-4715-8848-7f0ede97d571 None None] Registered counter for database nova_cell0 Oct 30 00:55:09.209581 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b80acdf6-3a3a-43eb-8479-b2cad9a13728 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:55:09.211563 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b80acdf6-3a3a-43eb-8479-b2cad9a13728 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:55:09.212589 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b80acdf6-3a3a-43eb-8479-b2cad9a13728 None None] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:55:09.213057 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b80acdf6-3a3a-43eb-8479-b2cad9a13728 None None] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:55:09.213947 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO dbcounter [None req-3a311086-d895-4715-8848-7f0ede97d571 None None] Registered counter for database nova_cell1 Oct 30 00:55:09.215079 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b80acdf6-3a3a-43eb-8479-b2cad9a13728 None None] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:55:09.217124 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO dbcounter [None req-b80acdf6-3a3a-43eb-8479-b2cad9a13728 None None] Registered counter for database nova_cell0 Oct 30 00:55:09.221179 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_db.sqlalchemy.engines [None req-3a311086-d895-4715-8848-7f0ede97d571 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=118961) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 30 00:55:09.221423 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO dbcounter [None req-b80acdf6-3a3a-43eb-8479-b2cad9a13728 None None] Registered counter for database nova_cell1 Oct 30 00:55:09.223875 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_db.sqlalchemy.engines [None req-3a311086-d895-4715-8848-7f0ede97d571 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=118961) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 30 00:55:09.228235 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writer thread running {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 30 00:55:09.229299 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_db.sqlalchemy.engines [None req-b80acdf6-3a3a-43eb-8479-b2cad9a13728 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=118962) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 30 00:55:09.231572 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_db.sqlalchemy.engines [None req-b80acdf6-3a3a-43eb-8479-b2cad9a13728 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=118962) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 30 00:55:09.236304 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writer thread running {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 30 00:55:09.239358 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writer thread running {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 30 00:55:09.369294 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writer thread running {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 30 00:55:09.378386 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO dbcounter [None req-3a311086-d895-4715-8848-7f0ede97d571 None None] Registered counter for database nova_cell0 Oct 30 00:55:09.386259 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_db.sqlalchemy.engines [None req-3a311086-d895-4715-8848-7f0ede97d571 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=118961) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 30 00:55:09.391394 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writer thread running {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 30 00:55:09.392104 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO dbcounter [None req-b80acdf6-3a3a-43eb-8479-b2cad9a13728 None None] Registered counter for database nova_cell0 Oct 30 00:55:09.399544 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_db.sqlalchemy.engines [None req-b80acdf6-3a3a-43eb-8479-b2cad9a13728 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=118962) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 30 00:55:09.404928 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writer thread running {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 30 00:55:09.717182 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: WARNING keystonemiddleware.auth_token [None req-3a311086-d895-4715-8848-7f0ede97d571 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 30 00:55:09.724518 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: WARNING keystonemiddleware.auth_token [None req-3a311086-d895-4715-8848-7f0ede97d571 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 30 00:55:09.736802 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: WARNING keystonemiddleware.auth_token [None req-b80acdf6-3a3a-43eb-8479-b2cad9a13728 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 30 00:55:09.744516 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: WARNING keystonemiddleware.auth_token [None req-b80acdf6-3a3a-43eb-8479-b2cad9a13728 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 30 00:55:10.078680 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: WARNING keystonemiddleware.auth_token [None req-3a311086-d895-4715-8848-7f0ede97d571 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 30 00:55:10.084917 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: WARNING keystonemiddleware.auth_token [None req-3a311086-d895-4715-8848-7f0ede97d571 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 30 00:55:10.090498 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5633d75709c0 pid: 118961 (default app) Oct 30 00:55:10.097694 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: WARNING keystonemiddleware.auth_token [None req-b80acdf6-3a3a-43eb-8479-b2cad9a13728 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 30 00:55:10.104383 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: WARNING keystonemiddleware.auth_token [None req-b80acdf6-3a3a-43eb-8479-b2cad9a13728 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 30 00:55:10.109744 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5633d75709c0 pid: 118962 (default app) Oct 30 00:55:46.373199 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_api:SELECT=3 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:55:46.374114 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell1:SELECT=3 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:55:46.375105 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell0:SELECT=3 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:55:46.376123 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell0:SELECT=3 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:55:47.745625 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-c40a6ef3-06ad-41c2-8d6d-edf60dfca99b tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] Action: 'create', calling method: .version_select at 0x7f0eb0b6fee0>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesSnapshotsTestJSON-volume-1173243418"}} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:55:48.986437 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-c40a6ef3-06ad-41c2-8d6d-edf60dfca99b tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] 10.222.0.16 "POST /compute/v2.1/os-volumes" status: 200 len: 345 microversion: 2.1 time: 2.623543 Oct 30 00:55:48.988475 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 1/1] 10.222.0.16 () {60 vars in 1155 bytes} [Sun Oct 30 00:55:46 2022] POST /compute/v2.1/os-volumes => generated 345 bytes in 2627 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) Oct 30 00:55:49.290491 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-27b88820-beaf-49eb-b7da-42fe0248ac89 tempest-VolumesTestJSON-1068182587 tempest-VolumesTestJSON-1068182587-project] Calling method '.version_select at 0x7f0eb09d6d30>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:55:49.448297 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-27b88820-beaf-49eb-b7da-42fe0248ac89 tempest-VolumesTestJSON-1068182587 tempest-VolumesTestJSON-1068182587-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/735a1ae5-f10e-4d84-8b28-3faa4b2ee041" status: 200 len: 351 microversion: 2.1 time: 0.148158 Oct 30 00:55:49.448297 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 2/2] 10.222.0.16 () {58 vars in 1245 bytes} [Sun Oct 30 00:55:49 2022] GET /compute/v2.1/os-volumes/735a1ae5-f10e-4d84-8b28-3faa4b2ee041 => generated 351 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:55:49.724022 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-7fc62446-60fb-4fff-be0f-737f4d573ac2 tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] Calling method '.version_select at 0x7f0eb0bb1a60>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:55:49.809315 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_api:SELECT=3 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:55:49.813290 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell1:SELECT=3 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:55:49.814227 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell0:SELECT=3 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:55:49.814976 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell0:SELECT=3 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:55:50.409554 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-7fc62446-60fb-4fff-be0f-737f4d573ac2 tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/058e9695-7f6c-4bb8-8659-159613431e0c" status: 200 len: 346 microversion: 2.1 time: 1.406574 Oct 30 00:55:50.410407 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 1/3] 10.222.0.16 () {58 vars in 1245 bytes} [Sun Oct 30 00:55:48 2022] GET /compute/v2.1/os-volumes/058e9695-7f6c-4bb8-8659-159613431e0c => generated 346 bytes in 1412 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:55:50.423966 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-622a2b4b-651c-412c-9eab-1afd36291e73 tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] Action: 'create', calling method: .version_select at 0x7f0eb096c430>, body: {"snapshot": {"volume_id": "058e9695-7f6c-4bb8-8659-159613431e0c", "display_name": "tempest-VolumesSnapshotsTestJSON-Snapshot-1777995878"}} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:55:50.995293 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-622a2b4b-651c-412c-9eab-1afd36291e73 tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] 10.222.0.16 "POST /compute/v2.1/os-snapshots" status: 200 len: 287 microversion: 2.1 time: 0.575499 Oct 30 00:55:50.996170 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 3/4] 10.222.0.16 () {60 vars in 1162 bytes} [Sun Oct 30 00:55:50 2022] POST /compute/v2.1/os-snapshots => generated 287 bytes in 577 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 30 00:55:51.009967 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-b99b490d-d463-48f7-836d-9526f96f00c5 tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] Calling method '.version_select at 0x7f0eb0a06d30>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:55:51.071259 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-b99b490d-d463-48f7-836d-9526f96f00c5 tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] 10.222.0.16 "GET /compute/v2.1/os-snapshots/654897a0-6c78-473d-b06d-c3eef7f56aab" status: 200 len: 287 microversion: 2.1 time: 0.067442 Oct 30 00:55:51.072049 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 2/5] 10.222.0.16 () {58 vars in 1251 bytes} [Sun Oct 30 00:55:51 2022] GET /compute/v2.1/os-snapshots/654897a0-6c78-473d-b06d-c3eef7f56aab => generated 287 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:55:51.085105 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-09b9d9fd-2e3a-405a-a1ed-7a52c8202707 tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] Calling method '.version_select at 0x7f0eb09725e0>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:55:51.211869 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-09b9d9fd-2e3a-405a-a1ed-7a52c8202707 tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] 10.222.0.16 "GET /compute/v2.1/os-snapshots" status: 200 len: 290 microversion: 2.1 time: 0.132173 Oct 30 00:55:51.212444 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 4/6] 10.222.0.16 () {58 vars in 1140 bytes} [Sun Oct 30 00:55:51 2022] GET /compute/v2.1/os-snapshots => generated 290 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:55:51.226540 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-068fcb3a-3fa3-4023-9021-1921c4feade7 tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] Calling method '.version_select at 0x7f0eb0a06ca0>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:55:51.367239 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-61b34236-a4d8-4832-87f6-cf92e2fa1406 tempest-VolumesTestJSON-1068182587 tempest-VolumesTestJSON-1068182587-project] Calling method '.version_select at 0x7f0eb096c280>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:55:51.376305 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-068fcb3a-3fa3-4023-9021-1921c4feade7 tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] 10.222.0.16 "GET /compute/v2.1/os-snapshots/654897a0-6c78-473d-b06d-c3eef7f56aab" status: 200 len: 287 microversion: 2.1 time: 0.155593 Oct 30 00:55:51.382441 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 3/7] 10.222.0.16 () {58 vars in 1251 bytes} [Sun Oct 30 00:55:51 2022] GET /compute/v2.1/os-snapshots/654897a0-6c78-473d-b06d-c3eef7f56aab => generated 287 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:55:51.475432 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-61b34236-a4d8-4832-87f6-cf92e2fa1406 tempest-VolumesTestJSON-1068182587 tempest-VolumesTestJSON-1068182587-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/e5581171-be60-4988-b1f9-31815ad0943d" status: 200 len: 350 microversion: 2.1 time: 0.117186 Oct 30 00:55:51.476822 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 5/8] 10.222.0.16 () {58 vars in 1245 bytes} [Sun Oct 30 00:55:51 2022] GET /compute/v2.1/os-volumes/e5581171-be60-4988-b1f9-31815ad0943d => generated 350 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 30 00:55:52.391187 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-d6864ce2-262f-411e-be91-ee43d3807905 tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] Calling method '.version_select at 0x7f0eb09d15e0>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:55:52.438913 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-d6864ce2-262f-411e-be91-ee43d3807905 tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] 10.222.0.16 "GET /compute/v2.1/os-snapshots/654897a0-6c78-473d-b06d-c3eef7f56aab" status: 200 len: 287 microversion: 2.1 time: 0.051811 Oct 30 00:55:52.439503 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 4/9] 10.222.0.16 () {58 vars in 1251 bytes} [Sun Oct 30 00:55:52 2022] GET /compute/v2.1/os-snapshots/654897a0-6c78-473d-b06d-c3eef7f56aab => generated 287 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:55:53.455569 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-b3ed7bba-a950-4a66-9619-d4f97d3cbaba tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] Calling method '.version_select at 0x7f0eb09d6c10>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:55:53.512608 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-b3ed7bba-a950-4a66-9619-d4f97d3cbaba tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] 10.222.0.16 "GET /compute/v2.1/os-snapshots/654897a0-6c78-473d-b06d-c3eef7f56aab" status: 200 len: 287 microversion: 2.1 time: 0.062623 Oct 30 00:55:53.514242 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 6/10] 10.222.0.16 () {58 vars in 1251 bytes} [Sun Oct 30 00:55:53 2022] GET /compute/v2.1/os-snapshots/654897a0-6c78-473d-b06d-c3eef7f56aab => generated 287 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:55:54.527982 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-69b40179-fc40-499b-a2f2-b208ad399261 tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] Calling method '.version_select at 0x7f0eb09d1ca0>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:55:54.584355 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-69b40179-fc40-499b-a2f2-b208ad399261 tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] 10.222.0.16 "GET /compute/v2.1/os-snapshots/654897a0-6c78-473d-b06d-c3eef7f56aab" status: 200 len: 287 microversion: 2.1 time: 0.060321 Oct 30 00:55:54.585751 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 5/11] 10.222.0.16 () {58 vars in 1251 bytes} [Sun Oct 30 00:55:54 2022] GET /compute/v2.1/os-snapshots/654897a0-6c78-473d-b06d-c3eef7f56aab => generated 287 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:55:55.598297 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-37a7868d-c336-4e4c-8a8e-4be7f8177560 tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] Calling method '.version_select at 0x7f0eb09720d0>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:55:55.647821 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-37a7868d-c336-4e4c-8a8e-4be7f8177560 tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] 10.222.0.16 "GET /compute/v2.1/os-snapshots/654897a0-6c78-473d-b06d-c3eef7f56aab" status: 200 len: 287 microversion: 2.1 time: 0.053341 Oct 30 00:55:55.648635 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 7/12] 10.222.0.16 () {58 vars in 1251 bytes} [Sun Oct 30 00:55:55 2022] GET /compute/v2.1/os-snapshots/654897a0-6c78-473d-b06d-c3eef7f56aab => generated 287 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:55:56.662837 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-9276ea93-4980-4917-b23d-07bccd7f6fa1 tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] Calling method '.version_select at 0x7f0eb09d1c10>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:55:56.755222 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-9276ea93-4980-4917-b23d-07bccd7f6fa1 tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] 10.222.0.16 "GET /compute/v2.1/os-snapshots/654897a0-6c78-473d-b06d-c3eef7f56aab" status: 200 len: 287 microversion: 2.1 time: 0.097739 Oct 30 00:55:56.764160 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 6/13] 10.222.0.16 () {58 vars in 1251 bytes} [Sun Oct 30 00:55:56 2022] GET /compute/v2.1/os-snapshots/654897a0-6c78-473d-b06d-c3eef7f56aab => generated 287 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:55:57.774510 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-5221cc47-48ad-4d97-812f-a2a4bb997b9e tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] Calling method '.version_select at 0x7f0eb0972160>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:55:57.868853 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-5221cc47-48ad-4d97-812f-a2a4bb997b9e tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] 10.222.0.16 "GET /compute/v2.1/os-snapshots/654897a0-6c78-473d-b06d-c3eef7f56aab" status: 200 len: 287 microversion: 2.1 time: 0.098936 Oct 30 00:55:57.870560 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 8/14] 10.222.0.16 () {58 vars in 1251 bytes} [Sun Oct 30 00:55:57 2022] GET /compute/v2.1/os-snapshots/654897a0-6c78-473d-b06d-c3eef7f56aab => generated 287 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:55:58.883142 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-3715c11c-75b0-48ba-96b5-84fb4059ebc9 tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] Calling method '.version_select at 0x7f0eb09d1550>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:55:58.954245 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-3715c11c-75b0-48ba-96b5-84fb4059ebc9 tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] 10.222.0.16 "GET /compute/v2.1/os-snapshots/654897a0-6c78-473d-b06d-c3eef7f56aab" status: 200 len: 288 microversion: 2.1 time: 0.075347 Oct 30 00:55:58.962000 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 7/15] 10.222.0.16 () {58 vars in 1251 bytes} [Sun Oct 30 00:55:58 2022] GET /compute/v2.1/os-snapshots/654897a0-6c78-473d-b06d-c3eef7f56aab => generated 288 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:55:58.969122 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-34e6abda-3a28-4450-ae43-cb98ed281f21 tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] Calling method '.version_select at 0x7f0eb0a06af0>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:55:59.123436 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-34e6abda-3a28-4450-ae43-cb98ed281f21 tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] 10.222.0.16 "DELETE /compute/v2.1/os-snapshots/654897a0-6c78-473d-b06d-c3eef7f56aab" status: 202 len: 0 microversion: 2.1 time: 0.158714 Oct 30 00:55:59.124009 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 8/16] 10.222.0.16 () {58 vars in 1254 bytes} [Sun Oct 30 00:55:58 2022] DELETE /compute/v2.1/os-snapshots/654897a0-6c78-473d-b06d-c3eef7f56aab => generated 0 bytes in 160 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 30 00:55:59.132382 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-8f3abb05-8da3-4e50-8a54-bb0b017d463d tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] Calling method '.version_select at 0x7f0eb0a06c10>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:55:59.201946 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-8f3abb05-8da3-4e50-8a54-bb0b017d463d tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] 10.222.0.16 "GET /compute/v2.1/os-snapshots/654897a0-6c78-473d-b06d-c3eef7f56aab" status: 200 len: 287 microversion: 2.1 time: 0.073470 Oct 30 00:55:59.202871 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 9/17] 10.222.0.16 () {58 vars in 1251 bytes} [Sun Oct 30 00:55:59 2022] GET /compute/v2.1/os-snapshots/654897a0-6c78-473d-b06d-c3eef7f56aab => generated 287 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 30 00:55:59.227329 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-c962517b-320b-44e8-ac1e-028d7900f7ad tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Action: 'create', calling method: .version_select at 0x7f0eb09d6b80>, body: {"keypair": {"name": "tempest-keypair-1396989907", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFy9ws4uVTpZ0yVJGICuuvXLbwANSHxbC1nA5R8BNcjft75KcnAiTBew+Rmm5QAwVAzCHOsg0tbFTy2r/7RzQlytEPAgO1bszVlal4qHNsYU1s2oYK4XAw8GvJP6dq47iA=="}} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:55:59.374818 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.quota [None req-c962517b-320b-44e8-ac1e-028d7900f7ad tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Getting quotas for project 6b68b886c19c45d585409651f33b8626. Resources: {'key_pairs'} {{(pid=118961) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 30 00:55:59.388151 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.quota [None req-c962517b-320b-44e8-ac1e-028d7900f7ad tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Getting quotas for user ecaf8ec105c04b919256f852da859eba and project 6b68b886c19c45d585409651f33b8626. Resources: {'key_pairs'} {{(pid=118961) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 30 00:55:59.491956 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-c962517b-320b-44e8-ac1e-028d7900f7ad tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.543272 Oct 30 00:55:59.492895 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 9/18] 10.222.0.16 () {60 vars in 1159 bytes} [Sun Oct 30 00:55:58 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 545 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 30 00:56:00.588188 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-c36e8956-b22e-4979-92a0-9b882db264d9 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-188306703", "imageRef": "4c16f4f6-84cb-4463-b82c-b7c65f3826f0", "flavorRef": "452", "networks": [{"uuid": "bb2af8ca-edb3-4bc3-b4f2-767946c55a59"}]}} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:56:00.600659 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-6733f642-8209-4d35-a521-cb4cdb0a0669 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Action: 'create', calling method: .version_select at 0x7f0eb09d1550>, body: {"keypair": {"name": "tempest-keypair-1492790923", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBERGfwaStIoSS4qGLnrREEe/gK+EYiCYqjCmosCrdimIF/J8TUiQAKLRE/5EvTULkdtVhqR0EL+tWhpQqQGOLMuR0Y6Mj4M2A/bgqN/vhLkrKMW9yxDGzHH12/6E/Zcqvg=="}} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:56:00.723967 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.quota [None req-6733f642-8209-4d35-a521-cb4cdb0a0669 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Getting quotas for project 8bf407954ed34ebfaf56db206df4fc84. Resources: {'key_pairs'} {{(pid=118962) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 30 00:56:00.738164 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.quota [None req-6733f642-8209-4d35-a521-cb4cdb0a0669 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Getting quotas for user 1c0e5bd11ac14259ad376ce54bcb7025 and project 8bf407954ed34ebfaf56db206df4fc84. Resources: {'key_pairs'} {{(pid=118962) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 30 00:56:00.788937 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.network.neutron [None req-c36e8956-b22e-4979-92a0-9b882db264d9 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] validate_networks() for [('bb2af8ca-edb3-4bc3-b4f2-767946c55a59', None, None, None, None, None)] {{(pid=118961) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 30 00:56:00.859331 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-6733f642-8209-4d35-a521-cb4cdb0a0669 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 1.015516 Oct 30 00:56:00.860036 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 10/19] 10.222.0.16 () {60 vars in 1159 bytes} [Sun Oct 30 00:55:59 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 1020 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:56:00.877712 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-6cf12894-ad83-4978-a0f5-8ec5d74527ec tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] Calling method '.version_select at 0x7f0eb09d19d0>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:00.952970 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-6cf12894-ad83-4978-a0f5-8ec5d74527ec tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] 10.222.0.16 "GET /compute/v2.1/os-snapshots/654897a0-6c78-473d-b06d-c3eef7f56aab" status: 200 len: 287 microversion: 2.1 time: 0.088388 Oct 30 00:56:00.953703 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 11/20] 10.222.0.16 () {58 vars in 1251 bytes} [Sun Oct 30 00:56:00 2022] GET /compute/v2.1/os-snapshots/654897a0-6c78-473d-b06d-c3eef7f56aab => generated 287 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:56:01.430201 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-5ab86005-8be3-443a-868b-a074ded35644 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Action: 'create', calling method: .version_select at 0x7f0eb09478b0>, body: {"keypair": {"name": "tempest-keypair-1576642984", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBEcdAemamoIs5Z88iZEniN09kGyhXigppXQaVTnsGrL87lt5hFNVsAUf4a8xaoz9VyV3mIZL/2BfbhakgFgNbrxdXE7sX4JpTcIsagEqhL8jO21L6OD2JqUkQ5k6E0aVIw=="}} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:56:01.472013 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.quota [None req-5ab86005-8be3-443a-868b-a074ded35644 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Getting quotas for project a3bfc7112bb04760aa4e9adc50b12361. Resources: {'key_pairs'} {{(pid=118962) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 30 00:56:01.476978 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.quota [None req-5ab86005-8be3-443a-868b-a074ded35644 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Getting quotas for user 15e603f9898a430d8a53f21818e97722 and project a3bfc7112bb04760aa4e9adc50b12361. Resources: {'key_pairs'} {{(pid=118962) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 30 00:56:01.517702 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-5ab86005-8be3-443a-868b-a074ded35644 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.20 time: 0.562183 Oct 30 00:56:01.518977 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 12/21] 10.222.0.16 () {62 vars in 1200 bytes} [Sun Oct 30 00:56:00 2022] POST /compute/v2.1/os-keypairs => generated 401 bytes in 565 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Oct 30 00:56:01.968931 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-ca6600a4-670f-4084-96f1-7bb46892b7e4 tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] Calling method '.version_select at 0x7f0eb08f7b80>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:02.038113 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-ca6600a4-670f-4084-96f1-7bb46892b7e4 tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] 10.222.0.16 "GET /compute/v2.1/os-snapshots/654897a0-6c78-473d-b06d-c3eef7f56aab" status: 200 len: 287 microversion: 2.1 time: 0.074596 Oct 30 00:56:02.038693 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 13/22] 10.222.0.16 () {58 vars in 1251 bytes} [Sun Oct 30 00:56:01 2022] GET /compute/v2.1/os-snapshots/654897a0-6c78-473d-b06d-c3eef7f56aab => generated 287 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:56:02.955805 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-c36e8956-b22e-4979-92a0-9b882db264d9 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Flavor limits 0:0:0 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 30 00:56:02.957267 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-c36e8956-b22e-4979-92a0-9b882db264d9 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Image limits 0:0:0 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 30 00:56:02.958712 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-c36e8956-b22e-4979-92a0-9b882db264d9 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Flavor pref 0:0:0 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 30 00:56:02.959569 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-c36e8956-b22e-4979-92a0-9b882db264d9 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Image pref 0:0:0 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 30 00:56:02.960427 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-c36e8956-b22e-4979-92a0-9b882db264d9 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 30 00:56:02.987132 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c36e8956-b22e-4979-92a0-9b882db264d9 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:02.990512 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c36e8956-b22e-4979-92a0-9b882db264d9 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:02.992946 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c36e8956-b22e-4979-92a0-9b882db264d9 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:02.998904 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c36e8956-b22e-4979-92a0-9b882db264d9 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:03.000837 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c36e8956-b22e-4979-92a0-9b882db264d9 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:03.001503 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c36e8956-b22e-4979-92a0-9b882db264d9 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:03.051084 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-8a3fc00c-9e6c-4090-9c4d-4421190f3b24 tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] Calling method '.version_select at 0x7f0eb08f81f0>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:03.102769 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.quota [None req-c36e8956-b22e-4979-92a0-9b882db264d9 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Getting quotas for project 0f7d505c7b9e4ca08d6437e6e529a039. Resources: {'ram', 'instances', 'cores'} {{(pid=118961) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 30 00:56:03.113958 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.quota [None req-c36e8956-b22e-4979-92a0-9b882db264d9 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Getting quotas for user c92f7b4d09d74b8d93d739c1938694e6 and project 0f7d505c7b9e4ca08d6437e6e529a039. Resources: {'ram', 'instances', 'cores'} {{(pid=118961) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 30 00:56:03.128077 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-8a3fc00c-9e6c-4090-9c4d-4421190f3b24 tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] 10.222.0.16 "GET /compute/v2.1/os-snapshots/654897a0-6c78-473d-b06d-c3eef7f56aab" status: 200 len: 287 microversion: 2.1 time: 0.081302 Oct 30 00:56:03.128750 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 14/23] 10.222.0.16 () {58 vars in 1251 bytes} [Sun Oct 30 00:56:03 2022] GET /compute/v2.1/os-snapshots/654897a0-6c78-473d-b06d-c3eef7f56aab => generated 287 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 30 00:56:03.150928 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-c36e8956-b22e-4979-92a0-9b882db264d9 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Going to run 1 instances... {{(pid=118961) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 30 00:56:03.160747 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-c36e8956-b22e-4979-92a0-9b882db264d9 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4c16f4f6-84cb-4463-b82c-b7c65f3826f0',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118961) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 30 00:56:03.365366 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-c36e8956-b22e-4979-92a0-9b882db264d9 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 3.190621 Oct 30 00:56:03.366581 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 10/24] 10.222.0.16 () {60 vars in 1147 bytes} [Sun Oct 30 00:56:00 2022] POST /compute/v2.1/servers => generated 380 bytes in 3192 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 30 00:56:03.385687 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-59d4b763-3e55-4229-803f-ead4608e6ea0 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:03.387444 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-59d4b763-3e55-4229-803f-ead4608e6ea0 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:03.525214 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-0fa9ed20-d1fa-4ec6-8075-25fa5f39373e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1041088324", "imageRef": "4c16f4f6-84cb-4463-b82c-b7c65f3826f0", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "4400f15d-ace0-4893-b5ff-c43f47dad5a5"}], "security_groups": [{"name": "tempest-securitygroup--685785836"}], "key_name": "tempest-keypair-1396989907", "user_data": "}} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:56:03.646821 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-59d4b763-3e55-4229-803f-ead4608e6ea0 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c92f7b4d09d74b8d93d739c1938694e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f7d505c7b9e4ca08d6437e6e529a039', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:03.696574 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-59d4b763-3e55-4229-803f-ead4608e6ea0 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] 10.222.0.16 "GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1" status: 200 len: 1214 microversion: 2.1 time: 0.318900 Oct 30 00:56:03.701426 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 15/25] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:03 2022] GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1 => generated 1214 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:03.772922 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.network.neutron [None req-0fa9ed20-d1fa-4ec6-8075-25fa5f39373e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] validate_networks() for [('4400f15d-ace0-4893-b5ff-c43f47dad5a5', None, None, None, None, None)] {{(pid=118961) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 30 00:56:04.144212 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-46a017f8-8a1e-487d-9d9b-d465831447b6 tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] Calling method '.version_select at 0x7f0eb08f73a0>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:04.226029 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.wsgi [None req-46a017f8-8a1e-487d-9d9b-d465831447b6 tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] HTTP exception thrown: Snapshot 654897a0-6c78-473d-b06d-c3eef7f56aab could not be found. Oct 30 00:56:04.231560 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-46a017f8-8a1e-487d-9d9b-d465831447b6 tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] Request method failure captured: Oct 30 00:56:04.231560 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: request: GET /compute/v2.1/os-snapshots/654897a0-6c78-473d-b06d-c3eef7f56aab HTTP/1.1 Oct 30 00:56:04.231560 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept: application/json Oct 30 00:56:04.231560 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept-Encoding: identity Oct 30 00:56:04.231560 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Connection: close Oct 30 00:56:04.231560 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Length: 0 Oct 30 00:56:04.231560 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Type: application/json Oct 30 00:56:04.231560 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Host: 10.222.0.16 Oct 30 00:56:04.231560 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Openstack-System-Scope: None Oct 30 00:56:04.231560 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: User-Agent: python-urllib3/1.26.12 Oct 30 00:56:04.231560 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Auth-Token: gAAAAABjXcuSXChPBMm-wKUoMvwBJeQGlzKYraVHw7G9HySbpQWpHokHbyKE6OvRTkUF2fl54AUiWUa5yYEErx78GM6bYsQz5sTKDP0dh5g7w_VRJ50wHy5qLEFPnIvG6TstD2HxCn3GEOB_XeisqnSrdnK-UNVEWQUe3GcfKg9UWsWgTyJrGJI Oct 30 00:56:04.231560 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Id: None Oct 30 00:56:04.231560 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Name: None Oct 30 00:56:04.231560 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Identity-Status: Confirmed Oct 30 00:56:04.231560 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Is-Admin-Project: True Oct 30 00:56:04.231560 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Id: default Oct 30 00:56:04.231560 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Name: Default Oct 30 00:56:04.231560 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Id: ce1df4d4d27a412b91845bc0fb02c3b9 Oct 30 00:56:04.231560 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1368085593 Oct 30 00:56:04.231560 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Role: reader,member Oct 30 00:56:04.231560 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Roles: reader,member Oct 30 00:56:04.233131 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_ce1df4d4d27a412b91845bc0fb02c3b9"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ce1df4d4d27a412b91845bc0fb02c3b9"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/ce1df4d4d27a412b91845bc0fb02c3b9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ce1df4d4d27a412b91845bc0fb02c3b9"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 00:56:04.233131 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1368085593 Oct 30 00:56:04.233131 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Id: ce1df4d4d27a412b91845bc0fb02c3b9 Oct 30 00:56:04.233131 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1368085593 Oct 30 00:56:04.233131 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User: tempest-VolumesSnapshotsTestJSON-1368085593-project Oct 30 00:56:04.233131 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Id: default Oct 30 00:56:04.233131 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Name: Default Oct 30 00:56:04.233131 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Id: 0038d0604f1942fdb1fcf91140d5197d Oct 30 00:56:04.233131 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1368085593-project Oct 30 00:56:04.233131 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: method: Oct 30 00:56:04.233131 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: exception: Snapshot 654897a0-6c78-473d-b06d-c3eef7f56aab could not be found. Oct 30 00:56:04.233131 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: action_args: {'id': '654897a0-6c78-473d-b06d-c3eef7f56aab'} Oct 30 00:56:04.233131 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 00:56:04.233987 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 00:56:04.233987 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Oct 30 00:56:04.233987 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Oct 30 00:56:04.233987 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Oct 30 00:56:04.233987 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Oct 30 00:56:04.233987 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Oct 30 00:56:04.233987 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Oct 30 00:56:04.233987 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 30 00:56:04.233987 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 30 00:56:04.233987 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 30 00:56:04.233987 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 30 00:56:04.233987 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 30 00:56:04.233987 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 30 00:56:04.233987 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 30 00:56:04.233987 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 30 00:56:04.233987 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Snapshot 654897a0-6c78-473d-b06d-c3eef7f56aab could not be found. (HTTP 404) (Request-ID: req-8d236e1c-a043-4032-a911-1a99e7c3223f) Oct 30 00:56:04.233987 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 00:56:04.233987 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 00:56:04.235679 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 00:56:04.235679 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 00:56:04.235679 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 617, in show Oct 30 00:56:04.235679 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get_snapshot(context, id) Oct 30 00:56:04.235679 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 30 00:56:04.235679 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 30 00:56:04.235679 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 466, in wrapper Oct 30 00:56:04.235679 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi _reraise(exception.SnapshotNotFound(snapshot_id=snapshot_id)) Oct 30 00:56:04.235679 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 30 00:56:04.235679 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 30 00:56:04.235679 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Oct 30 00:56:04.235679 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Oct 30 00:56:04.235679 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Oct 30 00:56:04.235679 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Oct 30 00:56:04.235679 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Oct 30 00:56:04.235679 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Oct 30 00:56:04.235679 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 30 00:56:04.235679 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 30 00:56:04.235679 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 30 00:56:04.266692 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 30 00:56:04.266692 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 30 00:56:04.266692 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 30 00:56:04.266692 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 30 00:56:04.266692 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 30 00:56:04.266692 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.exception.SnapshotNotFound: Snapshot 654897a0-6c78-473d-b06d-c3eef7f56aab could not be found. Oct 30 00:56:04.266692 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 00:56:04.266692 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 00:56:04.266692 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 00:56:04.266692 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 00:56:04.266692 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 00:56:04.266692 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 00:56:04.266692 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 00:56:04.266692 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 00:56:04.266692 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 30 00:56:04.266692 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 30 00:56:04.266692 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 00:56:04.266692 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 00:56:04.266692 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 619, in show Oct 30 00:56:04.266692 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 30 00:56:04.269370 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Snapshot 654897a0-6c78-473d-b06d-c3eef7f56aab could not be found. Oct 30 00:56:04.269370 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 00:56:04.269370 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 00:56:04.269370 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 00:56:04.269370 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 00:56:04.269370 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 00:56:04.269370 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 00:56:04.269370 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 00:56:04.269370 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 00:56:04.269370 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Snapshot 654897a0-6c78-473d-b06d-c3eef7f56aab could not be found. Oct 30 00:56:04.269370 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 00:56:04.269370 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-46a017f8-8a1e-487d-9d9b-d465831447b6 tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] Returning 404 to user: Snapshot 654897a0-6c78-473d-b06d-c3eef7f56aab could not be found. {{(pid=118962) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 00:56:04.269370 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-46a017f8-8a1e-487d-9d9b-d465831447b6 tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] 10.222.0.16 "GET /compute/v2.1/os-snapshots/654897a0-6c78-473d-b06d-c3eef7f56aab" status: 404 len: 111 microversion: 2.1 time: 0.094933 Oct 30 00:56:04.270258 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 16/26] 10.222.0.16 () {58 vars in 1251 bytes} [Sun Oct 30 00:56:04 2022] GET /compute/v2.1/os-snapshots/654897a0-6c78-473d-b06d-c3eef7f56aab => generated 111 bytes in 96 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 30 00:56:04.270258 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-2dddcd81-bc3f-4c04-abf8-deed06b76604 tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] Calling method '.version_select at 0x7f0eb087edc0>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:04.440425 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-2dddcd81-bc3f-4c04-abf8-deed06b76604 tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] 10.222.0.16 "DELETE /compute/v2.1/os-volumes/058e9695-7f6c-4bb8-8659-159613431e0c" status: 202 len: 0 microversion: 2.1 time: 0.201691 Oct 30 00:56:04.444945 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 17/27] 10.222.0.16 () {58 vars in 1248 bytes} [Sun Oct 30 00:56:04 2022] DELETE /compute/v2.1/os-volumes/058e9695-7f6c-4bb8-8659-159613431e0c => generated 0 bytes in 203 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 30 00:56:04.450982 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-b2640fd5-f508-4cde-8df0-98335633ab16 tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] Calling method '.version_select at 0x7f0eb0826550>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:04.710342 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-b2640fd5-f508-4cde-8df0-98335633ab16 tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/058e9695-7f6c-4bb8-8659-159613431e0c" status: 200 len: 345 microversion: 2.1 time: 0.263268 Oct 30 00:56:04.710942 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 18/28] 10.222.0.16 () {58 vars in 1245 bytes} [Sun Oct 30 00:56:04 2022] GET /compute/v2.1/os-volumes/058e9695-7f6c-4bb8-8659-159613431e0c => generated 345 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 30 00:56:04.716477 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-00c3240b-6dcc-4a90-8f8b-5481c3ec6d68 tempest-VolumesTestJSON-1068182587 tempest-VolumesTestJSON-1068182587-project] Calling method '.version_select at 0x7f0eb087eca0>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:04.851034 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-00c3240b-6dcc-4a90-8f8b-5481c3ec6d68 tempest-VolumesTestJSON-1068182587 tempest-VolumesTestJSON-1068182587-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/fdf843d0-5e39-436d-af0e-31b58d449b53" status: 200 len: 349 microversion: 2.1 time: 0.138812 Oct 30 00:56:04.851994 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 19/29] 10.222.0.16 () {58 vars in 1245 bytes} [Sun Oct 30 00:56:04 2022] GET /compute/v2.1/os-volumes/fdf843d0-5e39-436d-af0e-31b58d449b53 => generated 349 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:56:04.856547 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-86e1831d-74f8-48f6-ac7d-cff6ce7cb543 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:04.863584 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-86e1831d-74f8-48f6-ac7d-cff6ce7cb543 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:04.901536 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-0fa9ed20-d1fa-4ec6-8075-25fa5f39373e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Flavor limits 0:0:0 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 30 00:56:04.902492 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-0fa9ed20-d1fa-4ec6-8075-25fa5f39373e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Image limits 0:0:0 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 30 00:56:04.903002 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-0fa9ed20-d1fa-4ec6-8075-25fa5f39373e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Flavor pref 0:0:0 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 30 00:56:04.903414 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-0fa9ed20-d1fa-4ec6-8075-25fa5f39373e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Image pref 0:0:0 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 30 00:56:04.903846 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-0fa9ed20-d1fa-4ec6-8075-25fa5f39373e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 30 00:56:04.922393 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0fa9ed20-d1fa-4ec6-8075-25fa5f39373e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:04.923172 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0fa9ed20-d1fa-4ec6-8075-25fa5f39373e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:04.924132 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0fa9ed20-d1fa-4ec6-8075-25fa5f39373e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:04.925002 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0fa9ed20-d1fa-4ec6-8075-25fa5f39373e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:04.928343 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0fa9ed20-d1fa-4ec6-8075-25fa5f39373e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:04.929607 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0fa9ed20-d1fa-4ec6-8075-25fa5f39373e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:04.978169 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.quota [None req-0fa9ed20-d1fa-4ec6-8075-25fa5f39373e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Getting quotas for project 6b68b886c19c45d585409651f33b8626. Resources: {'ram', 'instances', 'cores'} {{(pid=118961) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 30 00:56:04.988860 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.quota [None req-0fa9ed20-d1fa-4ec6-8075-25fa5f39373e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Getting quotas for user ecaf8ec105c04b919256f852da859eba and project 6b68b886c19c45d585409651f33b8626. Resources: {'ram', 'instances', 'cores'} {{(pid=118961) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 30 00:56:05.002311 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-86e1831d-74f8-48f6-ac7d-cff6ce7cb543 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c92f7b4d09d74b8d93d739c1938694e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f7d505c7b9e4ca08d6437e6e529a039', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:05.016571 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-86e1831d-74f8-48f6-ac7d-cff6ce7cb543 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] 10.222.0.16 "GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1" status: 200 len: 1214 microversion: 2.1 time: 0.163502 Oct 30 00:56:05.020547 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 20/30] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:04 2022] GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1 => generated 1214 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:05.031108 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-647cfb55-fabb-4210-bdb7-953e0b28cd14 tempest-VolumesTestJSON-1068182587 tempest-VolumesTestJSON-1068182587-project] Calling method '.version_select at 0x7f0eb0826a60>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:05.037815 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-0fa9ed20-d1fa-4ec6-8075-25fa5f39373e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Going to run 1 instances... {{(pid=118961) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 30 00:56:05.040442 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-0fa9ed20-d1fa-4ec6-8075-25fa5f39373e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5652be64-c2a3-4b24-a3aa-c962f204c06e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4c16f4f6-84cb-4463-b82c-b7c65f3826f0',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118961) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 30 00:56:05.146742 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-0fa9ed20-d1fa-4ec6-8075-25fa5f39373e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 1.643656 Oct 30 00:56:05.148106 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 11/31] 10.222.0.16 () {60 vars in 1147 bytes} [Sun Oct 30 00:56:03 2022] POST /compute/v2.1/servers => generated 401 bytes in 1646 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 30 00:56:05.177528 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-86198d49-8173-45ed-b8c1-f672b84367cd tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:05.178912 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-86198d49-8173-45ed-b8c1-f672b84367cd tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5652be64-c2a3-4b24-a3aa-c962f204c06e] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:05.285224 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-647cfb55-fabb-4210-bdb7-953e0b28cd14 tempest-VolumesTestJSON-1068182587 tempest-VolumesTestJSON-1068182587-project] 10.222.0.16 "GET /compute/v2.1/os-volumes" status: 200 len: 1033 microversion: 2.1 time: 0.262661 Oct 30 00:56:05.285874 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 21/32] 10.222.0.16 () {58 vars in 1134 bytes} [Sun Oct 30 00:56:05 2022] GET /compute/v2.1/os-volumes => generated 1033 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:05.303127 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-9664a8b1-e8cd-42c6-81de-500f026d2905 tempest-VolumesTestJSON-1068182587 tempest-VolumesTestJSON-1068182587-project] Calling method '.version_select at 0x7f0eb0832040>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:05.429286 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-86198d49-8173-45ed-b8c1-f672b84367cd tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:05.512370 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-86198d49-8173-45ed-b8c1-f672b84367cd tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e" status: 200 len: 1242 microversion: 2.1 time: 0.342011 Oct 30 00:56:05.513288 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 12/33] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:05 2022] GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e => generated 1242 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:05.586911 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-9664a8b1-e8cd-42c6-81de-500f026d2905 tempest-VolumesTestJSON-1068182587 tempest-VolumesTestJSON-1068182587-project] 10.222.0.16 "GET /compute/v2.1/os-volumes?limit=2" status: 200 len: 692 microversion: 2.1 time: 0.289218 Oct 30 00:56:05.626153 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 22/34] 10.222.0.16 () {58 vars in 1149 bytes} [Sun Oct 30 00:56:05 2022] GET /compute/v2.1/os-volumes?limit=2 => generated 692 bytes in 330 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:56:05.627375 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-854e28fe-c750-484f-983e-905a69362ce6 tempest-VolumesTestJSON-1068182587 tempest-VolumesTestJSON-1068182587-project] Calling method '.version_select at 0x7f0eb08ae430>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:05.726310 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-56438388-70ad-44e0-9de1-4fcc524d1096 tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] Calling method '.version_select at 0x7f0eb0832940>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:05.819763 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.wsgi [None req-56438388-70ad-44e0-9de1-4fcc524d1096 tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] HTTP exception thrown: Volume 058e9695-7f6c-4bb8-8659-159613431e0c could not be found. Oct 30 00:56:05.822435 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-56438388-70ad-44e0-9de1-4fcc524d1096 tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] Request method failure captured: Oct 30 00:56:05.822435 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: request: GET /compute/v2.1/os-volumes/058e9695-7f6c-4bb8-8659-159613431e0c HTTP/1.1 Oct 30 00:56:05.822435 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept: application/json Oct 30 00:56:05.822435 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept-Encoding: identity Oct 30 00:56:05.822435 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Connection: close Oct 30 00:56:05.822435 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Length: 0 Oct 30 00:56:05.822435 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Type: application/json Oct 30 00:56:05.822435 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Host: 10.222.0.16 Oct 30 00:56:05.822435 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Openstack-System-Scope: None Oct 30 00:56:05.822435 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: User-Agent: python-urllib3/1.26.12 Oct 30 00:56:05.822435 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Auth-Token: gAAAAABjXcuSXChPBMm-wKUoMvwBJeQGlzKYraVHw7G9HySbpQWpHokHbyKE6OvRTkUF2fl54AUiWUa5yYEErx78GM6bYsQz5sTKDP0dh5g7w_VRJ50wHy5qLEFPnIvG6TstD2HxCn3GEOB_XeisqnSrdnK-UNVEWQUe3GcfKg9UWsWgTyJrGJI Oct 30 00:56:05.822435 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Id: None Oct 30 00:56:05.822435 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Name: None Oct 30 00:56:05.822435 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Identity-Status: Confirmed Oct 30 00:56:05.822435 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Is-Admin-Project: True Oct 30 00:56:05.822435 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Id: default Oct 30 00:56:05.822435 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Name: Default Oct 30 00:56:05.822435 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Id: ce1df4d4d27a412b91845bc0fb02c3b9 Oct 30 00:56:05.822435 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1368085593 Oct 30 00:56:05.822435 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Role: reader,member Oct 30 00:56:05.822435 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Roles: reader,member Oct 30 00:56:05.823700 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_ce1df4d4d27a412b91845bc0fb02c3b9"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ce1df4d4d27a412b91845bc0fb02c3b9"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/ce1df4d4d27a412b91845bc0fb02c3b9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ce1df4d4d27a412b91845bc0fb02c3b9"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 00:56:05.823700 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1368085593 Oct 30 00:56:05.823700 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Id: ce1df4d4d27a412b91845bc0fb02c3b9 Oct 30 00:56:05.823700 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1368085593 Oct 30 00:56:05.823700 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User: tempest-VolumesSnapshotsTestJSON-1368085593-project Oct 30 00:56:05.823700 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Id: default Oct 30 00:56:05.823700 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Name: Default Oct 30 00:56:05.823700 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Id: 0038d0604f1942fdb1fcf91140d5197d Oct 30 00:56:05.823700 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1368085593-project Oct 30 00:56:05.823700 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: method: Oct 30 00:56:05.823700 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: exception: Volume 058e9695-7f6c-4bb8-8659-159613431e0c could not be found. Oct 30 00:56:05.823700 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: action_args: {'id': '058e9695-7f6c-4bb8-8659-159613431e0c'} Oct 30 00:56:05.823700 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 00:56:05.824666 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 00:56:05.824666 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 30 00:56:05.824666 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 30 00:56:05.824666 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 30 00:56:05.824666 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 30 00:56:05.824666 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 30 00:56:05.824666 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 30 00:56:05.824666 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 30 00:56:05.824666 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 30 00:56:05.824666 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 30 00:56:05.824666 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 30 00:56:05.824666 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 30 00:56:05.824666 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 30 00:56:05.824666 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 30 00:56:05.824666 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 30 00:56:05.824666 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 058e9695-7f6c-4bb8-8659-159613431e0c could not be found. (HTTP 404) (Request-ID: req-721e2cb7-1bc0-489f-95ba-b476063a2cbf) Oct 30 00:56:05.824666 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 00:56:05.824666 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 00:56:05.824666 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 00:56:05.825673 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 00:56:05.825673 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Oct 30 00:56:05.825673 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Oct 30 00:56:05.825673 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 30 00:56:05.825673 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 30 00:56:05.825673 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 30 00:56:05.825673 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 30 00:56:05.825673 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 30 00:56:05.825673 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 30 00:56:05.825673 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 30 00:56:05.825673 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 30 00:56:05.825673 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 30 00:56:05.825673 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 30 00:56:05.825673 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 30 00:56:05.825673 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 30 00:56:05.825673 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 30 00:56:05.825673 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 30 00:56:05.825673 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 30 00:56:05.825673 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 30 00:56:05.826512 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 30 00:56:05.826512 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 30 00:56:05.826512 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 30 00:56:05.826512 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 30 00:56:05.826512 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 058e9695-7f6c-4bb8-8659-159613431e0c could not be found. Oct 30 00:56:05.826512 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 00:56:05.826512 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 00:56:05.826512 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 00:56:05.826512 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 00:56:05.826512 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 00:56:05.826512 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 00:56:05.826512 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 00:56:05.826512 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 00:56:05.826512 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 30 00:56:05.826512 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 30 00:56:05.826512 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 00:56:05.826512 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 00:56:05.826512 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Oct 30 00:56:05.826512 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 30 00:56:05.827534 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 058e9695-7f6c-4bb8-8659-159613431e0c could not be found. Oct 30 00:56:05.827534 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 00:56:05.827534 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 00:56:05.827534 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 00:56:05.827534 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 00:56:05.827534 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 00:56:05.827534 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 00:56:05.827534 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 00:56:05.827534 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 00:56:05.827534 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 058e9695-7f6c-4bb8-8659-159613431e0c could not be found. Oct 30 00:56:05.827534 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 00:56:05.829635 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-56438388-70ad-44e0-9de1-4fcc524d1096 tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] Returning 404 to user: Volume 058e9695-7f6c-4bb8-8659-159613431e0c could not be found. {{(pid=118962) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 00:56:05.838088 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-56438388-70ad-44e0-9de1-4fcc524d1096 tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/058e9695-7f6c-4bb8-8659-159613431e0c" status: 404 len: 109 microversion: 2.1 time: 0.116256 Oct 30 00:56:05.843520 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 23/35] 10.222.0.16 () {58 vars in 1245 bytes} [Sun Oct 30 00:56:05 2022] GET /compute/v2.1/os-volumes/058e9695-7f6c-4bb8-8659-159613431e0c => generated 109 bytes in 118 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 30 00:56:05.855091 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-966f8f01-1d8b-4f8b-a8d9-d70926d99392 tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] Calling method '.version_select at 0x7f0eb087e940>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:05.930848 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-854e28fe-c750-484f-983e-905a69362ce6 tempest-VolumesTestJSON-1068182587 tempest-VolumesTestJSON-1068182587-project] 10.222.0.16 "GET /compute/v2.1/os-volumes" status: 200 len: 1033 microversion: 2.1 time: 0.332201 Oct 30 00:56:05.939986 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 13/36] 10.222.0.16 () {58 vars in 1134 bytes} [Sun Oct 30 00:56:05 2022] GET /compute/v2.1/os-volumes => generated 1033 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:05.946608 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-abaab76d-643d-45ee-99aa-57bbf0408ae7 tempest-VolumesTestJSON-1068182587 tempest-VolumesTestJSON-1068182587-project] Calling method '.version_select at 0x7f0eabfcc8b0>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:05.977503 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.wsgi [None req-966f8f01-1d8b-4f8b-a8d9-d70926d99392 tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] HTTP exception thrown: Volume 058e9695-7f6c-4bb8-8659-159613431e0c could not be found. Oct 30 00:56:05.979644 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-966f8f01-1d8b-4f8b-a8d9-d70926d99392 tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] Request method failure captured: Oct 30 00:56:05.979644 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: request: DELETE /compute/v2.1/os-volumes/058e9695-7f6c-4bb8-8659-159613431e0c HTTP/1.1 Oct 30 00:56:05.979644 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept: application/json Oct 30 00:56:05.979644 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept-Encoding: identity Oct 30 00:56:05.979644 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Connection: close Oct 30 00:56:05.979644 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Length: 0 Oct 30 00:56:05.979644 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Type: application/json Oct 30 00:56:05.979644 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Host: 10.222.0.16 Oct 30 00:56:05.979644 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Openstack-System-Scope: None Oct 30 00:56:05.979644 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: User-Agent: python-urllib3/1.26.12 Oct 30 00:56:05.979644 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Auth-Token: gAAAAABjXcuSXChPBMm-wKUoMvwBJeQGlzKYraVHw7G9HySbpQWpHokHbyKE6OvRTkUF2fl54AUiWUa5yYEErx78GM6bYsQz5sTKDP0dh5g7w_VRJ50wHy5qLEFPnIvG6TstD2HxCn3GEOB_XeisqnSrdnK-UNVEWQUe3GcfKg9UWsWgTyJrGJI Oct 30 00:56:05.979644 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Id: None Oct 30 00:56:05.979644 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Name: None Oct 30 00:56:05.979644 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Identity-Status: Confirmed Oct 30 00:56:05.979644 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Is-Admin-Project: True Oct 30 00:56:05.979644 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Id: default Oct 30 00:56:05.979644 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Name: Default Oct 30 00:56:05.979644 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Id: ce1df4d4d27a412b91845bc0fb02c3b9 Oct 30 00:56:05.979644 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1368085593 Oct 30 00:56:05.979644 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Role: reader,member Oct 30 00:56:05.979644 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Roles: reader,member Oct 30 00:56:05.980952 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_ce1df4d4d27a412b91845bc0fb02c3b9"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ce1df4d4d27a412b91845bc0fb02c3b9"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/ce1df4d4d27a412b91845bc0fb02c3b9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ce1df4d4d27a412b91845bc0fb02c3b9"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 00:56:05.980952 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1368085593 Oct 30 00:56:05.980952 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Id: ce1df4d4d27a412b91845bc0fb02c3b9 Oct 30 00:56:05.980952 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1368085593 Oct 30 00:56:05.980952 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User: tempest-VolumesSnapshotsTestJSON-1368085593-project Oct 30 00:56:05.980952 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Id: default Oct 30 00:56:05.980952 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Name: Default Oct 30 00:56:05.980952 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Id: 0038d0604f1942fdb1fcf91140d5197d Oct 30 00:56:05.980952 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1368085593-project Oct 30 00:56:05.980952 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: method: Oct 30 00:56:05.980952 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: exception: Volume 058e9695-7f6c-4bb8-8659-159613431e0c could not be found. Oct 30 00:56:05.980952 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: action_args: {'id': '058e9695-7f6c-4bb8-8659-159613431e0c'} Oct 30 00:56:05.980952 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 00:56:05.981734 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 00:56:05.981734 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 30 00:56:05.981734 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 30 00:56:05.981734 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 30 00:56:05.981734 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 30 00:56:05.981734 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 30 00:56:05.981734 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 30 00:56:05.981734 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 30 00:56:05.981734 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 30 00:56:05.981734 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 30 00:56:05.981734 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 30 00:56:05.981734 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 30 00:56:05.981734 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 30 00:56:05.981734 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 30 00:56:05.981734 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 30 00:56:05.981734 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 058e9695-7f6c-4bb8-8659-159613431e0c could not be found. (HTTP 404) (Request-ID: req-cd1236ba-9ae4-4704-b123-010aea7b16b1) Oct 30 00:56:05.981734 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 00:56:05.981734 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 00:56:05.981734 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 00:56:05.982737 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 00:56:05.982737 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Oct 30 00:56:05.982737 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Oct 30 00:56:05.982737 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 30 00:56:05.982737 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 30 00:56:05.982737 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 30 00:56:05.982737 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 30 00:56:05.982737 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 30 00:56:05.982737 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 30 00:56:05.982737 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 30 00:56:05.982737 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 30 00:56:05.982737 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 30 00:56:05.982737 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 30 00:56:05.982737 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 30 00:56:05.982737 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 30 00:56:05.982737 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 30 00:56:05.982737 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 30 00:56:05.982737 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 30 00:56:05.982737 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 30 00:56:05.983745 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 30 00:56:05.983745 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 30 00:56:05.983745 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 30 00:56:05.983745 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 30 00:56:05.983745 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 058e9695-7f6c-4bb8-8659-159613431e0c could not be found. Oct 30 00:56:05.983745 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 00:56:05.983745 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 00:56:05.983745 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 00:56:05.983745 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 00:56:05.983745 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 00:56:05.983745 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 00:56:05.983745 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 00:56:05.983745 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 00:56:05.983745 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 30 00:56:05.983745 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 30 00:56:05.983745 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 00:56:05.983745 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 00:56:05.983745 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Oct 30 00:56:05.983745 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 30 00:56:05.985364 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 058e9695-7f6c-4bb8-8659-159613431e0c could not be found. Oct 30 00:56:05.985364 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 00:56:05.985364 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 00:56:05.985364 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 00:56:05.985364 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 00:56:05.985364 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 00:56:05.985364 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 00:56:05.985364 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 00:56:05.985364 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 00:56:05.985364 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 058e9695-7f6c-4bb8-8659-159613431e0c could not be found. Oct 30 00:56:05.985364 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 00:56:05.985364 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-966f8f01-1d8b-4f8b-a8d9-d70926d99392 tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] Returning 404 to user: Volume 058e9695-7f6c-4bb8-8659-159613431e0c could not be found. {{(pid=118962) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 00:56:05.985364 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-966f8f01-1d8b-4f8b-a8d9-d70926d99392 tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] 10.222.0.16 "DELETE /compute/v2.1/os-volumes/058e9695-7f6c-4bb8-8659-159613431e0c" status: 404 len: 109 microversion: 2.1 time: 0.135524 Oct 30 00:56:05.985364 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 24/37] 10.222.0.16 () {58 vars in 1248 bytes} [Sun Oct 30 00:56:05 2022] DELETE /compute/v2.1/os-volumes/058e9695-7f6c-4bb8-8659-159613431e0c => generated 109 bytes in 138 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 30 00:56:06.002795 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-617af564-ef2e-4f89-81e7-cf2dc906e2d3 tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] Calling method '.version_select at 0x7f0eb08324c0>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:06.149851 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-abaab76d-643d-45ee-99aa-57bbf0408ae7 tempest-VolumesTestJSON-1068182587 tempest-VolumesTestJSON-1068182587-project] 10.222.0.16 "GET /compute/v2.1/os-volumes?offset=1&limit=1" status: 200 len: 353 microversion: 2.1 time: 0.207850 Oct 30 00:56:06.151321 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 14/38] 10.222.0.16 () {58 vars in 1167 bytes} [Sun Oct 30 00:56:05 2022] GET /compute/v2.1/os-volumes?offset=1&limit=1 => generated 353 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:56:06.163228 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-b1f5d352-1987-4e06-ac0d-27ac2582acb5 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:06.163987 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-b1f5d352-1987-4e06-ac0d-27ac2582acb5 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:06.173628 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b1f5d352-1987-4e06-ac0d-27ac2582acb5 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:06.176310 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b1f5d352-1987-4e06-ac0d-27ac2582acb5 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:06.180456 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b1f5d352-1987-4e06-ac0d-27ac2582acb5 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:06.181849 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b1f5d352-1987-4e06-ac0d-27ac2582acb5 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:06.184005 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b1f5d352-1987-4e06-ac0d-27ac2582acb5 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:06.184673 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b1f5d352-1987-4e06-ac0d-27ac2582acb5 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:06.234160 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.wsgi [None req-617af564-ef2e-4f89-81e7-cf2dc906e2d3 tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] HTTP exception thrown: Volume 058e9695-7f6c-4bb8-8659-159613431e0c could not be found. Oct 30 00:56:06.237626 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-617af564-ef2e-4f89-81e7-cf2dc906e2d3 tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] Request method failure captured: Oct 30 00:56:06.237626 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: request: GET /compute/v2.1/os-volumes/058e9695-7f6c-4bb8-8659-159613431e0c HTTP/1.1 Oct 30 00:56:06.237626 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept: application/json Oct 30 00:56:06.237626 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept-Encoding: identity Oct 30 00:56:06.237626 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Connection: close Oct 30 00:56:06.237626 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Length: 0 Oct 30 00:56:06.237626 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Type: application/json Oct 30 00:56:06.237626 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Host: 10.222.0.16 Oct 30 00:56:06.237626 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Openstack-System-Scope: None Oct 30 00:56:06.237626 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: User-Agent: python-urllib3/1.26.12 Oct 30 00:56:06.237626 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Auth-Token: gAAAAABjXcuSXChPBMm-wKUoMvwBJeQGlzKYraVHw7G9HySbpQWpHokHbyKE6OvRTkUF2fl54AUiWUa5yYEErx78GM6bYsQz5sTKDP0dh5g7w_VRJ50wHy5qLEFPnIvG6TstD2HxCn3GEOB_XeisqnSrdnK-UNVEWQUe3GcfKg9UWsWgTyJrGJI Oct 30 00:56:06.237626 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Id: None Oct 30 00:56:06.237626 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Name: None Oct 30 00:56:06.237626 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Identity-Status: Confirmed Oct 30 00:56:06.237626 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Is-Admin-Project: True Oct 30 00:56:06.237626 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Id: default Oct 30 00:56:06.237626 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Name: Default Oct 30 00:56:06.237626 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Id: ce1df4d4d27a412b91845bc0fb02c3b9 Oct 30 00:56:06.237626 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1368085593 Oct 30 00:56:06.237626 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Role: reader,member Oct 30 00:56:06.237626 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Roles: reader,member Oct 30 00:56:06.238585 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_ce1df4d4d27a412b91845bc0fb02c3b9"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ce1df4d4d27a412b91845bc0fb02c3b9"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/ce1df4d4d27a412b91845bc0fb02c3b9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ce1df4d4d27a412b91845bc0fb02c3b9"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 00:56:06.238585 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1368085593 Oct 30 00:56:06.238585 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Id: ce1df4d4d27a412b91845bc0fb02c3b9 Oct 30 00:56:06.238585 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1368085593 Oct 30 00:56:06.238585 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User: tempest-VolumesSnapshotsTestJSON-1368085593-project Oct 30 00:56:06.238585 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Id: default Oct 30 00:56:06.238585 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Name: Default Oct 30 00:56:06.238585 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Id: 0038d0604f1942fdb1fcf91140d5197d Oct 30 00:56:06.238585 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1368085593-project Oct 30 00:56:06.238585 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: method: Oct 30 00:56:06.238585 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: exception: Volume 058e9695-7f6c-4bb8-8659-159613431e0c could not be found. Oct 30 00:56:06.238585 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: action_args: {'id': '058e9695-7f6c-4bb8-8659-159613431e0c'} Oct 30 00:56:06.238585 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 00:56:06.239303 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 00:56:06.239303 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 30 00:56:06.239303 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 30 00:56:06.239303 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 30 00:56:06.239303 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 30 00:56:06.239303 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 30 00:56:06.239303 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 30 00:56:06.239303 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 30 00:56:06.239303 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 30 00:56:06.239303 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 30 00:56:06.239303 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 30 00:56:06.239303 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 30 00:56:06.239303 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 30 00:56:06.239303 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 30 00:56:06.239303 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 30 00:56:06.239303 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 058e9695-7f6c-4bb8-8659-159613431e0c could not be found. (HTTP 404) (Request-ID: req-91ff00e2-7980-4169-8ce3-d7b92ccfee1b) Oct 30 00:56:06.239303 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 00:56:06.239303 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 00:56:06.239303 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 00:56:06.241097 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 00:56:06.241097 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Oct 30 00:56:06.241097 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Oct 30 00:56:06.241097 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 30 00:56:06.241097 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 30 00:56:06.241097 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 30 00:56:06.241097 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 30 00:56:06.241097 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 30 00:56:06.241097 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 30 00:56:06.241097 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 30 00:56:06.241097 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 30 00:56:06.241097 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 30 00:56:06.241097 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 30 00:56:06.241097 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 30 00:56:06.241097 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 30 00:56:06.241097 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 30 00:56:06.241097 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 30 00:56:06.241097 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 30 00:56:06.241097 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 30 00:56:06.241973 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 30 00:56:06.241973 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 30 00:56:06.241973 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 30 00:56:06.241973 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 30 00:56:06.241973 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 058e9695-7f6c-4bb8-8659-159613431e0c could not be found. Oct 30 00:56:06.241973 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 00:56:06.241973 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 00:56:06.241973 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 00:56:06.241973 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 00:56:06.241973 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 00:56:06.241973 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 00:56:06.241973 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 00:56:06.241973 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 00:56:06.241973 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 30 00:56:06.241973 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 30 00:56:06.241973 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 00:56:06.241973 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 00:56:06.241973 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Oct 30 00:56:06.241973 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 30 00:56:06.243921 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 058e9695-7f6c-4bb8-8659-159613431e0c could not be found. Oct 30 00:56:06.243921 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 00:56:06.243921 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 00:56:06.243921 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 00:56:06.243921 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 00:56:06.243921 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 00:56:06.243921 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 00:56:06.243921 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 00:56:06.243921 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 00:56:06.243921 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 058e9695-7f6c-4bb8-8659-159613431e0c could not be found. Oct 30 00:56:06.243921 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 00:56:06.243921 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-617af564-ef2e-4f89-81e7-cf2dc906e2d3 tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] Returning 404 to user: Volume 058e9695-7f6c-4bb8-8659-159613431e0c could not be found. {{(pid=118962) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 00:56:06.243921 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-617af564-ef2e-4f89-81e7-cf2dc906e2d3 tempest-VolumesSnapshotsTestJSON-1368085593 tempest-VolumesSnapshotsTestJSON-1368085593-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/058e9695-7f6c-4bb8-8659-159613431e0c" status: 404 len: 109 microversion: 2.1 time: 0.246261 Oct 30 00:56:06.243921 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 25/39] 10.222.0.16 () {58 vars in 1245 bytes} [Sun Oct 30 00:56:05 2022] GET /compute/v2.1/os-volumes/058e9695-7f6c-4bb8-8659-159613431e0c => generated 109 bytes in 254 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 30 00:56:06.248955 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-5d8a8879-4ef1-4595-8b6b-5f9482c16d55 tempest-VolumesTestJSON-1068182587 tempest-VolumesTestJSON-1068182587-project] Calling method '.version_select at 0x7f0eb087e9d0>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:06.574575 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-b1f5d352-1987-4e06-ac0d-27ac2582acb5 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c92f7b4d09d74b8d93d739c1938694e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f7d505c7b9e4ca08d6437e6e529a039', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:06.596061 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-b1f5d352-1987-4e06-ac0d-27ac2582acb5 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] 10.222.0.16 "GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1" status: 200 len: 1210 microversion: 2.1 time: 0.437034 Oct 30 00:56:06.598746 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 15/40] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:06 2022] GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1 => generated 1210 bytes in 446 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:06.608289 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-cafb06de-4a2e-451f-aa94-e08c7a455d34 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:06.609192 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-cafb06de-4a2e-451f-aa94-e08c7a455d34 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5652be64-c2a3-4b24-a3aa-c962f204c06e] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:06.640490 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-5d8a8879-4ef1-4595-8b6b-5f9482c16d55 tempest-VolumesTestJSON-1068182587 tempest-VolumesTestJSON-1068182587-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/detail?limit=2" status: 200 len: 692 microversion: 2.1 time: 0.398875 Oct 30 00:56:06.641086 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 26/41] 10.222.0.16 () {58 vars in 1170 bytes} [Sun Oct 30 00:56:06 2022] GET /compute/v2.1/os-volumes/detail?limit=2 => generated 692 bytes in 400 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:56:06.663112 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-cfd2d708-25ce-403e-966b-82168075b6fb tempest-VolumesTestJSON-1068182587 tempest-VolumesTestJSON-1068182587-project] Calling method '.version_select at 0x7f0eb0826ca0>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:06.784921 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-cafb06de-4a2e-451f-aa94-e08c7a455d34 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:06.793614 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-cafb06de-4a2e-451f-aa94-e08c7a455d34 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e" status: 200 len: 1242 microversion: 2.1 time: 0.193560 Oct 30 00:56:06.797818 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 16/42] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:06 2022] GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e => generated 1242 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:06.942649 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-802ba9e5-fe29-48fb-a307-9dbbea4bbdb6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-1625253882", "imageRef": "4c16f4f6-84cb-4463-b82c-b7c65f3826f0", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "8d646d29-e717-40c9-89d4-df0c802eb607"}], "security_groups": [{"name": "tempest-securitygroup--82360615"}], "key_name": "tempest-keypair-1576642984", "user_data": "}} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:56:07.072412 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-cfd2d708-25ce-403e-966b-82168075b6fb tempest-VolumesTestJSON-1068182587 tempest-VolumesTestJSON-1068182587-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1033 microversion: 2.1 time: 0.420738 Oct 30 00:56:07.073885 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 27/43] 10.222.0.16 () {58 vars in 1155 bytes} [Sun Oct 30 00:56:06 2022] GET /compute/v2.1/os-volumes/detail => generated 1033 bytes in 423 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 00:56:07.090113 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-cad53be7-29c6-492a-86f6-9247b7338fc0 tempest-VolumesTestJSON-1068182587 tempest-VolumesTestJSON-1068182587-project] Calling method '.version_select at 0x7f0eb0850700>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:07.241319 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.network.neutron [None req-802ba9e5-fe29-48fb-a307-9dbbea4bbdb6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] validate_networks() for [('8d646d29-e717-40c9-89d4-df0c802eb607', None, None, None, None, None)] {{(pid=118961) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 30 00:56:07.376395 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-cad53be7-29c6-492a-86f6-9247b7338fc0 tempest-VolumesTestJSON-1068182587 tempest-VolumesTestJSON-1068182587-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/detail?offset=1&limit=1" status: 200 len: 353 microversion: 2.1 time: 0.292312 Oct 30 00:56:07.377115 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 28/44] 10.222.0.16 () {58 vars in 1188 bytes} [Sun Oct 30 00:56:07 2022] GET /compute/v2.1/os-volumes/detail?offset=1&limit=1 => generated 353 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:56:07.394907 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-6176defb-4db1-4233-be00-14ec1e5461b3 tempest-VolumesTestJSON-1068182587 tempest-VolumesTestJSON-1068182587-project] Calling method '.version_select at 0x7f0eb0850820>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:07.782452 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-6176defb-4db1-4233-be00-14ec1e5461b3 tempest-VolumesTestJSON-1068182587 tempest-VolumesTestJSON-1068182587-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1033 microversion: 2.1 time: 0.393779 Oct 30 00:56:07.784216 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 29/45] 10.222.0.16 () {58 vars in 1155 bytes} [Sun Oct 30 00:56:07 2022] GET /compute/v2.1/os-volumes/detail => generated 1033 bytes in 396 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:07.790325 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-45248440-b08c-4fa9-9eb9-5511bde73024 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:07.791915 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-45248440-b08c-4fa9-9eb9-5511bde73024 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:07.841703 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-45248440-b08c-4fa9-9eb9-5511bde73024 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:07.842532 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-45248440-b08c-4fa9-9eb9-5511bde73024 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:07.844444 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-45248440-b08c-4fa9-9eb9-5511bde73024 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:07.852492 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-45248440-b08c-4fa9-9eb9-5511bde73024 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:07.855415 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-45248440-b08c-4fa9-9eb9-5511bde73024 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:07.858236 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-45248440-b08c-4fa9-9eb9-5511bde73024 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:08.383920 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-45248440-b08c-4fa9-9eb9-5511bde73024 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c92f7b4d09d74b8d93d739c1938694e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f7d505c7b9e4ca08d6437e6e529a039', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:08.413842 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-45248440-b08c-4fa9-9eb9-5511bde73024 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] 10.222.0.16 "GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1" status: 200 len: 1274 microversion: 2.1 time: 0.627816 Oct 30 00:56:08.418530 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 30/46] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:07 2022] GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1 => generated 1274 bytes in 633 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:08.436113 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-978ffab2-a1c1-4684-b945-d9fd22058f18 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:08.438894 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-978ffab2-a1c1-4684-b945-d9fd22058f18 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5652be64-c2a3-4b24-a3aa-c962f204c06e] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:08.460112 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-978ffab2-a1c1-4684-b945-d9fd22058f18 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:08.461241 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-978ffab2-a1c1-4684-b945-d9fd22058f18 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:08.461890 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-978ffab2-a1c1-4684-b945-d9fd22058f18 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:08.462805 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-978ffab2-a1c1-4684-b945-d9fd22058f18 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:08.463529 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-978ffab2-a1c1-4684-b945-d9fd22058f18 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:08.464263 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-978ffab2-a1c1-4684-b945-d9fd22058f18 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:08.585249 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-978ffab2-a1c1-4684-b945-d9fd22058f18 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:08.597344 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-978ffab2-a1c1-4684-b945-d9fd22058f18 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e" status: 200 len: 1294 microversion: 2.1 time: 0.173521 Oct 30 00:56:08.614752 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 31/47] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:08 2022] GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e => generated 1294 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:08.637387 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-6edd0571-a4b9-4c3c-a416-f63b12678681 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-1498008727", "imageRef": "4c16f4f6-84cb-4463-b82c-b7c65f3826f0", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "ccfb740b-9abc-42f4-8eda-aa2c80b4bdb0"}], "security_groups": [{"name": "tempest-securitygroup--1731801722"}], "key_name": "tempest-keypair-1492790923", "user_data": "}} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:56:08.730970 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-802ba9e5-fe29-48fb-a307-9dbbea4bbdb6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Flavor limits 0:0:0 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 30 00:56:08.731445 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-802ba9e5-fe29-48fb-a307-9dbbea4bbdb6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Image limits 0:0:0 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 30 00:56:08.731998 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-802ba9e5-fe29-48fb-a307-9dbbea4bbdb6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Flavor pref 0:0:0 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 30 00:56:08.732406 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-802ba9e5-fe29-48fb-a307-9dbbea4bbdb6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Image pref 0:0:0 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 30 00:56:08.733025 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-802ba9e5-fe29-48fb-a307-9dbbea4bbdb6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 30 00:56:08.755188 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-802ba9e5-fe29-48fb-a307-9dbbea4bbdb6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:08.755693 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-802ba9e5-fe29-48fb-a307-9dbbea4bbdb6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:08.756229 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-802ba9e5-fe29-48fb-a307-9dbbea4bbdb6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:08.756988 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-802ba9e5-fe29-48fb-a307-9dbbea4bbdb6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:08.757444 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-802ba9e5-fe29-48fb-a307-9dbbea4bbdb6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:08.757872 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-802ba9e5-fe29-48fb-a307-9dbbea4bbdb6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:08.819822 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.quota [None req-802ba9e5-fe29-48fb-a307-9dbbea4bbdb6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Getting quotas for project a3bfc7112bb04760aa4e9adc50b12361. Resources: {'ram', 'instances', 'cores'} {{(pid=118961) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 30 00:56:08.829038 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.quota [None req-802ba9e5-fe29-48fb-a307-9dbbea4bbdb6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Getting quotas for user 15e603f9898a430d8a53f21818e97722 and project a3bfc7112bb04760aa4e9adc50b12361. Resources: {'ram', 'instances', 'cores'} {{(pid=118961) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 30 00:56:08.861879 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-802ba9e5-fe29-48fb-a307-9dbbea4bbdb6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Going to run 1 instances... {{(pid=118961) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 30 00:56:08.869417 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-802ba9e5-fe29-48fb-a307-9dbbea4bbdb6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4c16f4f6-84cb-4463-b82c-b7c65f3826f0',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118961) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 30 00:56:08.975336 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-802ba9e5-fe29-48fb-a307-9dbbea4bbdb6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 400 microversion: 2.20 time: 2.036584 Oct 30 00:56:08.989510 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 17/48] 10.222.0.16 () {62 vars in 1188 bytes} [Sun Oct 30 00:56:06 2022] POST /compute/v2.1/servers => generated 400 bytes in 2052 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Oct 30 00:56:08.996959 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-193b15e2-96e5-4432-8c9f-29610a1b4a14 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:08.998811 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-193b15e2-96e5-4432-8c9f-29610a1b4a14 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:09.452877 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-193b15e2-96e5-4432-8c9f-29610a1b4a14 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:09.466357 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-193b15e2-96e5-4432-8c9f-29610a1b4a14 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:09.486926 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-193b15e2-96e5-4432-8c9f-29610a1b4a14 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:09.488153 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-193b15e2-96e5-4432-8c9f-29610a1b4a14 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1282 microversion: 2.20 time: 0.497312 Oct 30 00:56:09.488743 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 18/49] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 00:56:08 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1282 bytes in 499 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 00:56:09.533247 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.network.neutron [None req-6edd0571-a4b9-4c3c-a416-f63b12678681 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] validate_networks() for [('ccfb740b-9abc-42f4-8eda-aa2c80b4bdb0', None, None, None, None, None)] {{(pid=118962) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 30 00:56:09.541631 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-7a3619a5-9086-4e2c-b753-06e5c99aa7ba tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:09.549468 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-7a3619a5-9086-4e2c-b753-06e5c99aa7ba tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:09.595683 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7a3619a5-9086-4e2c-b753-06e5c99aa7ba tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:09.598912 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7a3619a5-9086-4e2c-b753-06e5c99aa7ba tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:09.599396 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7a3619a5-9086-4e2c-b753-06e5c99aa7ba tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:09.602805 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7a3619a5-9086-4e2c-b753-06e5c99aa7ba tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:09.603326 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7a3619a5-9086-4e2c-b753-06e5c99aa7ba tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:09.605210 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7a3619a5-9086-4e2c-b753-06e5c99aa7ba tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:10.024880 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-7a3619a5-9086-4e2c-b753-06e5c99aa7ba tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c92f7b4d09d74b8d93d739c1938694e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f7d505c7b9e4ca08d6437e6e529a039', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:10.042575 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-7a3619a5-9086-4e2c-b753-06e5c99aa7ba tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] 10.222.0.16 "GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1" status: 200 len: 1272 microversion: 2.1 time: 0.550005 Oct 30 00:56:10.044371 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 19/50] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:09 2022] GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1 => generated 1272 bytes in 554 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:10.054552 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-d7ff9288-5534-4c4c-a342-8e2a7a552b88 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:10.059377 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-d7ff9288-5534-4c4c-a342-8e2a7a552b88 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5652be64-c2a3-4b24-a3aa-c962f204c06e] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:10.074389 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d7ff9288-5534-4c4c-a342-8e2a7a552b88 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:10.075250 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d7ff9288-5534-4c4c-a342-8e2a7a552b88 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:10.075869 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d7ff9288-5534-4c4c-a342-8e2a7a552b88 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:10.083072 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d7ff9288-5534-4c4c-a342-8e2a7a552b88 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:10.083770 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d7ff9288-5534-4c4c-a342-8e2a7a552b88 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:10.084711 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d7ff9288-5534-4c4c-a342-8e2a7a552b88 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:10.626938 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-d7ff9288-5534-4c4c-a342-8e2a7a552b88 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:10.642370 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-d7ff9288-5534-4c4c-a342-8e2a7a552b88 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e" status: 200 len: 1300 microversion: 2.1 time: 0.595022 Oct 30 00:56:10.642960 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 20/51] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:10 2022] GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e => generated 1300 bytes in 598 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:10.659109 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-e5ad898a-d5b8-4b25-b49f-8c715140d2c4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:10.660999 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-e5ad898a-d5b8-4b25-b49f-8c715140d2c4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:10.699709 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e5ad898a-d5b8-4b25-b49f-8c715140d2c4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:10.703144 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e5ad898a-d5b8-4b25-b49f-8c715140d2c4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:10.704390 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e5ad898a-d5b8-4b25-b49f-8c715140d2c4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:10.707080 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e5ad898a-d5b8-4b25-b49f-8c715140d2c4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:10.716952 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e5ad898a-d5b8-4b25-b49f-8c715140d2c4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:10.722476 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e5ad898a-d5b8-4b25-b49f-8c715140d2c4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:11.027970 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-e5ad898a-d5b8-4b25-b49f-8c715140d2c4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:11.048655 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-e5ad898a-d5b8-4b25-b49f-8c715140d2c4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:11.061077 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-e5ad898a-d5b8-4b25-b49f-8c715140d2c4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:11.065376 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-e5ad898a-d5b8-4b25-b49f-8c715140d2c4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1334 microversion: 2.20 time: 0.417453 Oct 30 00:56:11.070707 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 21/52] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 00:56:10 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1334 bytes in 424 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 00:56:11.102729 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-09805446-1518-4f32-9d89-976296e9f58b tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:11.105376 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-09805446-1518-4f32-9d89-976296e9f58b tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:11.164034 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-09805446-1518-4f32-9d89-976296e9f58b tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:11.173701 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-09805446-1518-4f32-9d89-976296e9f58b tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.016s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:11.177334 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-09805446-1518-4f32-9d89-976296e9f58b tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:11.178339 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-09805446-1518-4f32-9d89-976296e9f58b tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:11.178823 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-09805446-1518-4f32-9d89-976296e9f58b tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:11.179264 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-09805446-1518-4f32-9d89-976296e9f58b tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:11.533681 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-09805446-1518-4f32-9d89-976296e9f58b tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c92f7b4d09d74b8d93d739c1938694e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f7d505c7b9e4ca08d6437e6e529a039', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:11.587973 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-09805446-1518-4f32-9d89-976296e9f58b tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] 10.222.0.16 "GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1" status: 200 len: 1314 microversion: 2.1 time: 0.482003 Oct 30 00:56:11.587973 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 22/53] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:11 2022] GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1 => generated 1314 bytes in 513 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:11.668595 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-3a1e4073-1311-4cd8-ac3e-8e4ad2764a80 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:11.669440 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-3a1e4073-1311-4cd8-ac3e-8e4ad2764a80 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5652be64-c2a3-4b24-a3aa-c962f204c06e] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:11.682997 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3a1e4073-1311-4cd8-ac3e-8e4ad2764a80 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:11.684506 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3a1e4073-1311-4cd8-ac3e-8e4ad2764a80 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:11.685018 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3a1e4073-1311-4cd8-ac3e-8e4ad2764a80 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:11.687558 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3a1e4073-1311-4cd8-ac3e-8e4ad2764a80 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:11.689526 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3a1e4073-1311-4cd8-ac3e-8e4ad2764a80 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:11.691202 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3a1e4073-1311-4cd8-ac3e-8e4ad2764a80 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:12.228763 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.virt.hardware [None req-6edd0571-a4b9-4c3c-a416-f63b12678681 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Flavor limits 0:0:0 {{(pid=118962) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 30 00:56:12.232122 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.virt.hardware [None req-6edd0571-a4b9-4c3c-a416-f63b12678681 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Image limits 0:0:0 {{(pid=118962) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 30 00:56:12.234681 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.virt.hardware [None req-6edd0571-a4b9-4c3c-a416-f63b12678681 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Flavor pref 0:0:0 {{(pid=118962) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 30 00:56:12.235320 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.virt.hardware [None req-6edd0571-a4b9-4c3c-a416-f63b12678681 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Image pref 0:0:0 {{(pid=118962) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 30 00:56:12.236641 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.virt.hardware [None req-6edd0571-a4b9-4c3c-a416-f63b12678681 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118962) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 30 00:56:12.258403 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6edd0571-a4b9-4c3c-a416-f63b12678681 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:12.265764 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6edd0571-a4b9-4c3c-a416-f63b12678681 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:12.266487 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6edd0571-a4b9-4c3c-a416-f63b12678681 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:12.268429 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6edd0571-a4b9-4c3c-a416-f63b12678681 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:12.269312 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6edd0571-a4b9-4c3c-a416-f63b12678681 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:12.270117 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6edd0571-a4b9-4c3c-a416-f63b12678681 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:12.312786 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.quota [None req-6edd0571-a4b9-4c3c-a416-f63b12678681 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Getting quotas for project 8bf407954ed34ebfaf56db206df4fc84. Resources: {'ram', 'instances', 'cores'} {{(pid=118962) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 30 00:56:12.318834 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.quota [None req-6edd0571-a4b9-4c3c-a416-f63b12678681 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Getting quotas for user 1c0e5bd11ac14259ad376ce54bcb7025 and project 8bf407954ed34ebfaf56db206df4fc84. Resources: {'ram', 'instances', 'cores'} {{(pid=118962) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 30 00:56:12.337514 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-6edd0571-a4b9-4c3c-a416-f63b12678681 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Going to run 1 instances... {{(pid=118962) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 30 00:56:12.341325 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-6edd0571-a4b9-4c3c-a416-f63b12678681 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4c16f4f6-84cb-4463-b82c-b7c65f3826f0',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118962) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 30 00:56:12.424745 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-3a1e4073-1311-4cd8-ac3e-8e4ad2764a80 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:12.433257 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-3a1e4073-1311-4cd8-ac3e-8e4ad2764a80 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e" status: 200 len: 1300 microversion: 2.1 time: 0.769661 Oct 30 00:56:12.433831 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 23/54] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:11 2022] GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e => generated 1300 bytes in 772 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:12.457693 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-9d9afa6c-d5b0-402d-b1fd-ffce8ac77d76 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:12.458318 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-9d9afa6c-d5b0-402d-b1fd-ffce8ac77d76 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:12.496969 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9d9afa6c-d5b0-402d-b1fd-ffce8ac77d76 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:12.497473 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9d9afa6c-d5b0-402d-b1fd-ffce8ac77d76 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:12.497918 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9d9afa6c-d5b0-402d-b1fd-ffce8ac77d76 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:12.498645 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9d9afa6c-d5b0-402d-b1fd-ffce8ac77d76 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:12.499116 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9d9afa6c-d5b0-402d-b1fd-ffce8ac77d76 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:12.499537 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9d9afa6c-d5b0-402d-b1fd-ffce8ac77d76 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:12.501426 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-6edd0571-a4b9-4c3c-a416-f63b12678681 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 3.884834 Oct 30 00:56:12.514820 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 32/55] 10.222.0.16 () {60 vars in 1147 bytes} [Sun Oct 30 00:56:08 2022] POST /compute/v2.1/servers => generated 402 bytes in 3899 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 30 00:56:12.536847 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-a2760a5b-a083-42c6-934c-792967edb0f9 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:12.540841 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-a2760a5b-a083-42c6-934c-792967edb0f9 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:13.371354 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-a2760a5b-a083-42c6-934c-792967edb0f9 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:13.380345 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-a2760a5b-a083-42c6-934c-792967edb0f9 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1238 microversion: 2.1 time: 0.856574 Oct 30 00:56:13.383177 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 33/56] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:12 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1238 bytes in 861 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:13.389028 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-d27db63b-892a-488f-802a-f8a4eb5701f6 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:13.393592 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-d27db63b-892a-488f-802a-f8a4eb5701f6 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:13.413634 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d27db63b-892a-488f-802a-f8a4eb5701f6 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:13.414469 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d27db63b-892a-488f-802a-f8a4eb5701f6 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:13.415113 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d27db63b-892a-488f-802a-f8a4eb5701f6 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:13.416528 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d27db63b-892a-488f-802a-f8a4eb5701f6 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:13.425824 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d27db63b-892a-488f-802a-f8a4eb5701f6 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:13.429241 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d27db63b-892a-488f-802a-f8a4eb5701f6 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:13.519294 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-9d9afa6c-d5b0-402d-b1fd-ffce8ac77d76 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:13.530821 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-9d9afa6c-d5b0-402d-b1fd-ffce8ac77d76 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:13.542661 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-9d9afa6c-d5b0-402d-b1fd-ffce8ac77d76 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:13.545680 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-9d9afa6c-d5b0-402d-b1fd-ffce8ac77d76 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1340 microversion: 2.20 time: 1.104846 Oct 30 00:56:13.550198 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 24/57] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 00:56:12 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1340 bytes in 1110 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 00:56:13.564488 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-14268725-2011-439d-9a92-78d88683ef04 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:13.565401 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-14268725-2011-439d-9a92-78d88683ef04 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5652be64-c2a3-4b24-a3aa-c962f204c06e] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:13.581613 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-14268725-2011-439d-9a92-78d88683ef04 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:13.582220 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-14268725-2011-439d-9a92-78d88683ef04 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:13.582848 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-14268725-2011-439d-9a92-78d88683ef04 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:13.583815 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-14268725-2011-439d-9a92-78d88683ef04 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:13.584567 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-14268725-2011-439d-9a92-78d88683ef04 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:13.585235 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-14268725-2011-439d-9a92-78d88683ef04 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:14.624564 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-14268725-2011-439d-9a92-78d88683ef04 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:14.632612 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-d27db63b-892a-488f-802a-f8a4eb5701f6 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c92f7b4d09d74b8d93d739c1938694e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f7d505c7b9e4ca08d6437e6e529a039', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:14.641854 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-d27db63b-892a-488f-802a-f8a4eb5701f6 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] 10.222.0.16 "GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1" status: 200 len: 1314 microversion: 2.1 time: 1.256649 Oct 30 00:56:14.649039 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-14268725-2011-439d-9a92-78d88683ef04 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e" status: 200 len: 1367 microversion: 2.1 time: 1.096723 Oct 30 00:56:14.653544 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 25/58] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:13 2022] GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e => generated 1367 bytes in 1103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:14.664359 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 34/59] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:13 2022] GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1 => generated 1314 bytes in 1279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:14.673758 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-389526a0-5ed9-4593-a931-0769b6b8627d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:14.696585 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-8a0c647e-cc07-4a70-8fb2-8e59e4621979 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:14.696585 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-8a0c647e-cc07-4a70-8fb2-8e59e4621979 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:14.697829 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-389526a0-5ed9-4593-a931-0769b6b8627d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:14.716251 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-389526a0-5ed9-4593-a931-0769b6b8627d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:14.718764 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-389526a0-5ed9-4593-a931-0769b6b8627d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:14.720291 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-389526a0-5ed9-4593-a931-0769b6b8627d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:14.729442 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-389526a0-5ed9-4593-a931-0769b6b8627d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:14.730122 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-389526a0-5ed9-4593-a931-0769b6b8627d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:14.734135 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-389526a0-5ed9-4593-a931-0769b6b8627d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:14.742443 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8a0c647e-cc07-4a70-8fb2-8e59e4621979 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:14.749041 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8a0c647e-cc07-4a70-8fb2-8e59e4621979 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:14.752339 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8a0c647e-cc07-4a70-8fb2-8e59e4621979 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:14.758965 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8a0c647e-cc07-4a70-8fb2-8e59e4621979 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:14.761905 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8a0c647e-cc07-4a70-8fb2-8e59e4621979 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:14.763082 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8a0c647e-cc07-4a70-8fb2-8e59e4621979 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:15.456110 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-8a0c647e-cc07-4a70-8fb2-8e59e4621979 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:15.466762 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-389526a0-5ed9-4593-a931-0769b6b8627d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:15.472430 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-8a0c647e-cc07-4a70-8fb2-8e59e4621979 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1234 microversion: 2.1 time: 0.814641 Oct 30 00:56:15.474004 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 26/60] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:14 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1234 bytes in 819 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:15.481268 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-389526a0-5ed9-4593-a931-0769b6b8627d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:15.482764 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-389526a0-5ed9-4593-a931-0769b6b8627d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:15.483968 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-389526a0-5ed9-4593-a931-0769b6b8627d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1340 microversion: 2.20 time: 0.814198 Oct 30 00:56:15.500124 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 35/61] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 00:56:14 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1340 bytes in 835 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 00:56:15.681532 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-1b2b048b-29ab-4e35-a0df-40f22166961e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:15.683856 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-1b2b048b-29ab-4e35-a0df-40f22166961e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5652be64-c2a3-4b24-a3aa-c962f204c06e] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:15.707494 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1b2b048b-29ab-4e35-a0df-40f22166961e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:15.714035 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1b2b048b-29ab-4e35-a0df-40f22166961e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:15.717003 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1b2b048b-29ab-4e35-a0df-40f22166961e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:15.728789 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1b2b048b-29ab-4e35-a0df-40f22166961e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:15.729579 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1b2b048b-29ab-4e35-a0df-40f22166961e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:15.730191 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1b2b048b-29ab-4e35-a0df-40f22166961e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:16.223292 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-70acece6-f231-4447-a0df-3201a75abb79 tempest-VolumesGetTestJSON-1986195394 tempest-VolumesGetTestJSON-1986195394-project] Action: 'create', calling method: .version_select at 0x7f0eabf70b80>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesGetTestJSON-Volume-530139786", "metadata": {"Type": "work"}}} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:56:16.332251 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-1b2b048b-29ab-4e35-a0df-40f22166961e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:16.361399 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-1b2b048b-29ab-4e35-a0df-40f22166961e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e" status: 200 len: 1367 microversion: 2.1 time: 0.684758 Oct 30 00:56:16.385775 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 36/62] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:15 2022] GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e => generated 1367 bytes in 711 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:16.396487 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-07a6ca33-966a-4632-b71b-3eb01b5b54a5 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:16.398036 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-07a6ca33-966a-4632-b71b-3eb01b5b54a5 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:16.410254 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-07a6ca33-966a-4632-b71b-3eb01b5b54a5 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:16.412343 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-07a6ca33-966a-4632-b71b-3eb01b5b54a5 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:16.413090 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-07a6ca33-966a-4632-b71b-3eb01b5b54a5 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:16.414624 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-07a6ca33-966a-4632-b71b-3eb01b5b54a5 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:16.415446 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-07a6ca33-966a-4632-b71b-3eb01b5b54a5 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:16.416391 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-07a6ca33-966a-4632-b71b-3eb01b5b54a5 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:17.139531 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-70acece6-f231-4447-a0df-3201a75abb79 tempest-VolumesGetTestJSON-1986195394 tempest-VolumesGetTestJSON-1986195394-project] 10.222.0.16 "POST /compute/v2.1/os-volumes" status: 200 len: 352 microversion: 2.1 time: 1.663859 Oct 30 00:56:17.140675 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 27/63] 10.222.0.16 () {60 vars in 1156 bytes} [Sun Oct 30 00:56:15 2022] POST /compute/v2.1/os-volumes => generated 352 bytes in 1666 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) Oct 30 00:56:17.156875 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-93958aea-a12f-423c-aa1d-d426228b4cb1 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:17.157526 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-93958aea-a12f-423c-aa1d-d426228b4cb1 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:17.187237 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-93958aea-a12f-423c-aa1d-d426228b4cb1 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:17.187757 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-93958aea-a12f-423c-aa1d-d426228b4cb1 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:17.188205 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-93958aea-a12f-423c-aa1d-d426228b4cb1 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:17.188922 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-93958aea-a12f-423c-aa1d-d426228b4cb1 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:17.189376 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-93958aea-a12f-423c-aa1d-d426228b4cb1 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:17.189796 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-93958aea-a12f-423c-aa1d-d426228b4cb1 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:17.483813 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-93958aea-a12f-423c-aa1d-d426228b4cb1 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:17.491676 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-93958aea-a12f-423c-aa1d-d426228b4cb1 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1296 microversion: 2.1 time: 0.349852 Oct 30 00:56:17.504835 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 28/64] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:17 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1296 bytes in 363 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:17.510139 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-341ffe4f-aaf2-4a48-a2e7-7a0c21bc9bec tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:17.511078 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-341ffe4f-aaf2-4a48-a2e7-7a0c21bc9bec tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:17.521867 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-07a6ca33-966a-4632-b71b-3eb01b5b54a5 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c92f7b4d09d74b8d93d739c1938694e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f7d505c7b9e4ca08d6437e6e529a039', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:17.523147 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-341ffe4f-aaf2-4a48-a2e7-7a0c21bc9bec tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:17.523880 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-341ffe4f-aaf2-4a48-a2e7-7a0c21bc9bec tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:17.524536 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-341ffe4f-aaf2-4a48-a2e7-7a0c21bc9bec tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:17.525450 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-341ffe4f-aaf2-4a48-a2e7-7a0c21bc9bec tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:17.526109 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-341ffe4f-aaf2-4a48-a2e7-7a0c21bc9bec tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:17.526964 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-341ffe4f-aaf2-4a48-a2e7-7a0c21bc9bec tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:17.533085 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-07a6ca33-966a-4632-b71b-3eb01b5b54a5 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] 10.222.0.16 "GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1" status: 200 len: 1314 microversion: 2.1 time: 1.143245 Oct 30 00:56:17.533672 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 37/65] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:16 2022] GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1 => generated 1314 bytes in 1147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:17.544190 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-e5c33aba-a745-4d07-9b2a-a8ccfc2c947a tempest-VolumesGetTestJSON-1986195394 tempest-VolumesGetTestJSON-1986195394-project] Calling method '.version_select at 0x7f0eb07134c0>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:17.714301 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-e5c33aba-a745-4d07-9b2a-a8ccfc2c947a tempest-VolumesGetTestJSON-1986195394 tempest-VolumesGetTestJSON-1986195394-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/a487f977-045f-450d-b387-d91e31419efe" status: 200 len: 352 microversion: 2.1 time: 0.178864 Oct 30 00:56:17.721330 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 38/66] 10.222.0.16 () {58 vars in 1245 bytes} [Sun Oct 30 00:56:17 2022] GET /compute/v2.1/os-volumes/a487f977-045f-450d-b387-d91e31419efe => generated 352 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:56:17.721330 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-bdb9b3b0-3d6e-4cc8-b1be-aa5b0f5c6b2e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:17.721594 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-bdb9b3b0-3d6e-4cc8-b1be-aa5b0f5c6b2e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5652be64-c2a3-4b24-a3aa-c962f204c06e] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:17.735403 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bdb9b3b0-3d6e-4cc8-b1be-aa5b0f5c6b2e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:17.735928 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bdb9b3b0-3d6e-4cc8-b1be-aa5b0f5c6b2e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:17.736638 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bdb9b3b0-3d6e-4cc8-b1be-aa5b0f5c6b2e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:17.737674 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bdb9b3b0-3d6e-4cc8-b1be-aa5b0f5c6b2e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:17.738449 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bdb9b3b0-3d6e-4cc8-b1be-aa5b0f5c6b2e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:17.739062 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bdb9b3b0-3d6e-4cc8-b1be-aa5b0f5c6b2e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:18.073779 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-341ffe4f-aaf2-4a48-a2e7-7a0c21bc9bec tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:18.091143 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-341ffe4f-aaf2-4a48-a2e7-7a0c21bc9bec tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:18.093133 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-341ffe4f-aaf2-4a48-a2e7-7a0c21bc9bec tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:18.094461 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-341ffe4f-aaf2-4a48-a2e7-7a0c21bc9bec tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1406 microversion: 2.20 time: 0.588338 Oct 30 00:56:18.108140 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 29/67] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 00:56:17 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1406 bytes in 602 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 00:56:18.423094 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [req-f832d78e-f60e-46e8-9913-d42d7024029b req-df7b2b0a-3827-4b15-bda7-7bbafdfe7959 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0f858c8d-6414-403e-b8d3-3ecd728e84a1", "tag": "2c0a4a56-006d-4324-b26e-f48bac7f7a5b"}]} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:56:18.459654 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_api:SELECT=90,nova_api:INSERT=10 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:56:18.479961 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-f832d78e-f60e-46e8-9913-d42d7024029b req-df7b2b0a-3827-4b15-bda7-7bbafdfe7959 service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:18.480617 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-f832d78e-f60e-46e8-9913-d42d7024029b req-df7b2b0a-3827-4b15-bda7-7bbafdfe7959 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:18.481797 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-f832d78e-f60e-46e8-9913-d42d7024029b req-df7b2b0a-3827-4b15-bda7-7bbafdfe7959 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:18.485803 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-bdb9b3b0-3d6e-4cc8-b1be-aa5b0f5c6b2e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:18.509703 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-bdb9b3b0-3d6e-4cc8-b1be-aa5b0f5c6b2e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e" status: 200 len: 1367 microversion: 2.1 time: 0.787372 Oct 30 00:56:18.511693 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 39/68] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:17 2022] GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e => generated 1367 bytes in 796 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:18.543135 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-2d8d2e4e-079c-4137-96a0-6d3551e3835a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:18.545217 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.compute.server_external_events [req-f832d78e-f60e-46e8-9913-d42d7024029b req-df7b2b0a-3827-4b15-bda7-7bbafdfe7959 service nova] Creating event network-changed:2c0a4a56-006d-4324-b26e-f48bac7f7a5b for instance 0f858c8d-6414-403e-b8d3-3ecd728e84a1 on hv-cinder-maste Oct 30 00:56:18.546067 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-2d8d2e4e-079c-4137-96a0-6d3551e3835a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:18.547482 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-f832d78e-f60e-46e8-9913-d42d7024029b req-df7b2b0a-3827-4b15-bda7-7bbafdfe7959 service nova] Acquiring lock "compute-rpcapi-router" {{(pid=118961) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Oct 30 00:56:18.547917 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-f832d78e-f60e-46e8-9913-d42d7024029b req-df7b2b0a-3827-4b15-bda7-7bbafdfe7959 service nova] Acquired lock "compute-rpcapi-router" {{(pid=118961) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Oct 30 00:56:18.551835 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ca874cae-99b0-45b1-9d2d-26f0b103e609 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:18.552424 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ca874cae-99b0-45b1-9d2d-26f0b103e609 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:18.552885 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ca874cae-99b0-45b1-9d2d-26f0b103e609 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:18.555964 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ca874cae-99b0-45b1-9d2d-26f0b103e609 None None] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:18.557587 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ca874cae-99b0-45b1-9d2d-26f0b103e609 None None] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:18.558070 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ca874cae-99b0-45b1-9d2d-26f0b103e609 None None] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:18.566403 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2d8d2e4e-079c-4137-96a0-6d3551e3835a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:18.568503 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2d8d2e4e-079c-4137-96a0-6d3551e3835a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:18.570407 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2d8d2e4e-079c-4137-96a0-6d3551e3835a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:18.571495 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2d8d2e4e-079c-4137-96a0-6d3551e3835a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:18.572430 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2d8d2e4e-079c-4137-96a0-6d3551e3835a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:18.573984 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2d8d2e4e-079c-4137-96a0-6d3551e3835a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:18.633884 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.compute.rpcapi [None req-ca874cae-99b0-45b1-9d2d-26f0b103e609 None None] Automatically selected compute RPC version 6.1 from minimum service version 64 Oct 30 00:56:18.636998 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ca874cae-99b0-45b1-9d2d-26f0b103e609 None None] Releasing lock "compute-rpcapi-router" {{(pid=118961) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Oct 30 00:56:18.686203 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-ca874cae-99b0-45b1-9d2d-26f0b103e609 None None] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.576135 Oct 30 00:56:18.694953 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 30/69] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 00:56:18 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 586 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:56:18.701136 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-7c6167e2-f43f-4628-9491-f038a1f55f43 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:18.703843 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-7c6167e2-f43f-4628-9491-f038a1f55f43 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:18.714380 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7c6167e2-f43f-4628-9491-f038a1f55f43 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:18.715605 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7c6167e2-f43f-4628-9491-f038a1f55f43 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:18.716537 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7c6167e2-f43f-4628-9491-f038a1f55f43 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:18.717566 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7c6167e2-f43f-4628-9491-f038a1f55f43 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:18.718373 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7c6167e2-f43f-4628-9491-f038a1f55f43 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:18.719225 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7c6167e2-f43f-4628-9491-f038a1f55f43 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:18.943442 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-2d8d2e4e-079c-4137-96a0-6d3551e3835a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:18.968031 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-2d8d2e4e-079c-4137-96a0-6d3551e3835a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1296 microversion: 2.1 time: 0.431255 Oct 30 00:56:18.978260 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 40/70] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:18 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1296 bytes in 453 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 00:56:19.001831 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-a2663b69-1cb9-452c-bceb-f4b45de15afd tempest-VolumesGetTestJSON-1986195394 tempest-VolumesGetTestJSON-1986195394-project] Calling method '.version_select at 0x7f0eb0713820>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:19.238713 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-a2663b69-1cb9-452c-bceb-f4b45de15afd tempest-VolumesGetTestJSON-1986195394 tempest-VolumesGetTestJSON-1986195394-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/a487f977-045f-450d-b387-d91e31419efe" status: 200 len: 353 microversion: 2.1 time: 0.257453 Oct 30 00:56:19.239818 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 41/71] 10.222.0.16 () {58 vars in 1245 bytes} [Sun Oct 30 00:56:18 2022] GET /compute/v2.1/os-volumes/a487f977-045f-450d-b387-d91e31419efe => generated 353 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:56:19.251139 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-e5766eec-7670-4f16-8625-b4507708d5da tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:19.251944 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-e5766eec-7670-4f16-8625-b4507708d5da tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:19.296098 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e5766eec-7670-4f16-8625-b4507708d5da tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:19.299308 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e5766eec-7670-4f16-8625-b4507708d5da tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:19.301257 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e5766eec-7670-4f16-8625-b4507708d5da tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:19.307768 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e5766eec-7670-4f16-8625-b4507708d5da tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:19.308591 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e5766eec-7670-4f16-8625-b4507708d5da tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:19.311674 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e5766eec-7670-4f16-8625-b4507708d5da tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:19.458290 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell0:SELECT=28 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:56:19.793799 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-7c6167e2-f43f-4628-9491-f038a1f55f43 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c92f7b4d09d74b8d93d739c1938694e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f7d505c7b9e4ca08d6437e6e529a039', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:19.807769 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-7c6167e2-f43f-4628-9491-f038a1f55f43 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] 10.222.0.16 "GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1" status: 200 len: 1314 microversion: 2.1 time: 1.111013 Oct 30 00:56:19.808653 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 31/72] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:18 2022] GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1 => generated 1314 bytes in 1113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:19.824726 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-7824df43-76b2-4395-ac41-975827318a0f tempest-VolumesGetTestJSON-1986195394 tempest-VolumesGetTestJSON-1986195394-project] Calling method '.version_select at 0x7f0eb083f040>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:19.917543 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-e5766eec-7670-4f16-8625-b4507708d5da tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:19.956053 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-e5766eec-7670-4f16-8625-b4507708d5da tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:19.958664 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-7824df43-76b2-4395-ac41-975827318a0f tempest-VolumesGetTestJSON-1986195394 tempest-VolumesGetTestJSON-1986195394-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/a487f977-045f-450d-b387-d91e31419efe" status: 200 len: 353 microversion: 2.1 time: 0.147529 Oct 30 00:56:19.959697 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 32/73] 10.222.0.16 () {58 vars in 1245 bytes} [Sun Oct 30 00:56:19 2022] GET /compute/v2.1/os-volumes/a487f977-045f-450d-b387-d91e31419efe => generated 353 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:56:19.971138 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-e5766eec-7670-4f16-8625-b4507708d5da tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:19.982707 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-e5766eec-7670-4f16-8625-b4507708d5da tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1406 microversion: 2.20 time: 0.736818 Oct 30 00:56:19.986114 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 42/74] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 00:56:19 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1406 bytes in 745 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 00:56:20.015088 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-9494e5f7-2a76-4d27-bb90-a6cd7c2abdfb tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:20.016056 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-9494e5f7-2a76-4d27-bb90-a6cd7c2abdfb tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5652be64-c2a3-4b24-a3aa-c962f204c06e] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:20.030898 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9494e5f7-2a76-4d27-bb90-a6cd7c2abdfb tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:20.031423 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9494e5f7-2a76-4d27-bb90-a6cd7c2abdfb tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:20.031910 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9494e5f7-2a76-4d27-bb90-a6cd7c2abdfb tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:20.033558 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9494e5f7-2a76-4d27-bb90-a6cd7c2abdfb tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:20.034084 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9494e5f7-2a76-4d27-bb90-a6cd7c2abdfb tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:20.034463 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9494e5f7-2a76-4d27-bb90-a6cd7c2abdfb tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:20.480936 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [req-fbe803be-67c2-45e6-a7b9-2a9f6cc674b1 req-b3ec69c0-4eed-4ec6-93df-a67ac12c296f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5652be64-c2a3-4b24-a3aa-c962f204c06e", "tag": "9218be5d-9ff8-4e15-b073-e87bc5e0401c"}]} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:56:20.494886 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-fbe803be-67c2-45e6-a7b9-2a9f6cc674b1 req-b3ec69c0-4eed-4ec6-93df-a67ac12c296f service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:20.495948 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-fbe803be-67c2-45e6-a7b9-2a9f6cc674b1 req-b3ec69c0-4eed-4ec6-93df-a67ac12c296f service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:20.496777 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-fbe803be-67c2-45e6-a7b9-2a9f6cc674b1 req-b3ec69c0-4eed-4ec6-93df-a67ac12c296f service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:20.511378 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.compute.server_external_events [req-fbe803be-67c2-45e6-a7b9-2a9f6cc674b1 req-b3ec69c0-4eed-4ec6-93df-a67ac12c296f service nova] Creating event network-changed:9218be5d-9ff8-4e15-b073-e87bc5e0401c for instance 5652be64-c2a3-4b24-a3aa-c962f204c06e on hv-cinder-maste Oct 30 00:56:20.519970 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [req-fbe803be-67c2-45e6-a7b9-2a9f6cc674b1 req-b3ec69c0-4eed-4ec6-93df-a67ac12c296f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.558447 Oct 30 00:56:20.521200 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 33/75] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 00:56:19 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 561 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:56:20.528903 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-2b141223-edf9-4e49-a30b-3c76f2932dac tempest-VolumesGetTestJSON-1986195394 tempest-VolumesGetTestJSON-1986195394-project] Calling method '.version_select at 0x7f0eabf18af0>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:20.688358 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-2b141223-edf9-4e49-a30b-3c76f2932dac tempest-VolumesGetTestJSON-1986195394 tempest-VolumesGetTestJSON-1986195394-project] 10.222.0.16 "DELETE /compute/v2.1/os-volumes/a487f977-045f-450d-b387-d91e31419efe" status: 202 len: 0 microversion: 2.1 time: 0.165387 Oct 30 00:56:20.689026 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 34/76] 10.222.0.16 () {58 vars in 1248 bytes} [Sun Oct 30 00:56:20 2022] DELETE /compute/v2.1/os-volumes/a487f977-045f-450d-b387-d91e31419efe => generated 0 bytes in 167 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 30 00:56:20.694006 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-52fa214b-266c-402c-9838-efacc8a03315 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:20.695210 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-52fa214b-266c-402c-9838-efacc8a03315 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:20.710241 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-52fa214b-266c-402c-9838-efacc8a03315 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:20.712148 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-52fa214b-266c-402c-9838-efacc8a03315 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:20.712681 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-52fa214b-266c-402c-9838-efacc8a03315 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:20.713682 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-52fa214b-266c-402c-9838-efacc8a03315 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:20.715548 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-52fa214b-266c-402c-9838-efacc8a03315 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:20.716846 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-52fa214b-266c-402c-9838-efacc8a03315 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:21.032073 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-52fa214b-266c-402c-9838-efacc8a03315 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:21.041283 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-9494e5f7-2a76-4d27-bb90-a6cd7c2abdfb tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:21.058403 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-9494e5f7-2a76-4d27-bb90-a6cd7c2abdfb tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e" status: 200 len: 1367 microversion: 2.1 time: 1.068970 Oct 30 00:56:21.059384 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 43/77] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:19 2022] GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e => generated 1367 bytes in 1072 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:21.064465 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-52fa214b-266c-402c-9838-efacc8a03315 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1296 microversion: 2.1 time: 0.374132 Oct 30 00:56:21.084116 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 35/78] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:20 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1296 bytes in 394 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:21.092089 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-da22c444-7c33-463f-9f89-589d13df12b5 tempest-VolumesGetTestJSON-1986195394 tempest-VolumesGetTestJSON-1986195394-project] Calling method '.version_select at 0x7f0eafea0a60>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:21.101704 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-e9e3040e-9476-45d7-af0f-9e4483a67bdc tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:21.106129 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-e9e3040e-9476-45d7-af0f-9e4483a67bdc tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:21.143083 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e9e3040e-9476-45d7-af0f-9e4483a67bdc tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:21.144587 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e9e3040e-9476-45d7-af0f-9e4483a67bdc tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:21.151217 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e9e3040e-9476-45d7-af0f-9e4483a67bdc tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:21.152713 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e9e3040e-9476-45d7-af0f-9e4483a67bdc tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:21.154929 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e9e3040e-9476-45d7-af0f-9e4483a67bdc tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:21.155574 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e9e3040e-9476-45d7-af0f-9e4483a67bdc tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:21.181115 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.wsgi [None req-da22c444-7c33-463f-9f89-589d13df12b5 tempest-VolumesGetTestJSON-1986195394 tempest-VolumesGetTestJSON-1986195394-project] HTTP exception thrown: Volume a487f977-045f-450d-b387-d91e31419efe could not be found. Oct 30 00:56:21.188249 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-da22c444-7c33-463f-9f89-589d13df12b5 tempest-VolumesGetTestJSON-1986195394 tempest-VolumesGetTestJSON-1986195394-project] Request method failure captured: Oct 30 00:56:21.188249 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: request: GET /compute/v2.1/os-volumes/a487f977-045f-450d-b387-d91e31419efe HTTP/1.1 Oct 30 00:56:21.188249 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept: application/json Oct 30 00:56:21.188249 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept-Encoding: identity Oct 30 00:56:21.188249 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Connection: close Oct 30 00:56:21.188249 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Length: 0 Oct 30 00:56:21.188249 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Type: application/json Oct 30 00:56:21.188249 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Host: 10.222.0.16 Oct 30 00:56:21.188249 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Openstack-System-Scope: None Oct 30 00:56:21.188249 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: User-Agent: python-urllib3/1.26.12 Oct 30 00:56:21.188249 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Auth-Token: gAAAAABjXcuuJOv487GAMszxNFnI-Qa1iybRwK5ONWVur36NVKdM--HIUdlSPoCRIZRz8TeYTo1pAmQKrWVOlw5wg6x7HljYc_3CxkesXHCFVzQ1sptSl_tHer2J2Ilgv0JkKNjpDBOweOUhSDKQX9_mEerIoHP_p_xpoqHPrjx_2fVT3MSq28g Oct 30 00:56:21.188249 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Id: None Oct 30 00:56:21.188249 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Name: None Oct 30 00:56:21.188249 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Identity-Status: Confirmed Oct 30 00:56:21.188249 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Is-Admin-Project: True Oct 30 00:56:21.188249 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Id: default Oct 30 00:56:21.188249 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Name: Default Oct 30 00:56:21.188249 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Id: 782a4eade27d42298bbf4519f07d3e7f Oct 30 00:56:21.188249 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Name: tempest-VolumesGetTestJSON-1986195394 Oct 30 00:56:21.188249 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Role: reader,member Oct 30 00:56:21.188249 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Roles: reader,member Oct 30 00:56:21.190286 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_782a4eade27d42298bbf4519f07d3e7f"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/782a4eade27d42298bbf4519f07d3e7f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/782a4eade27d42298bbf4519f07d3e7f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/782a4eade27d42298bbf4519f07d3e7f"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 00:56:21.190286 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant: tempest-VolumesGetTestJSON-1986195394 Oct 30 00:56:21.190286 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Id: 782a4eade27d42298bbf4519f07d3e7f Oct 30 00:56:21.190286 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Name: tempest-VolumesGetTestJSON-1986195394 Oct 30 00:56:21.190286 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User: tempest-VolumesGetTestJSON-1986195394-project Oct 30 00:56:21.190286 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Id: default Oct 30 00:56:21.190286 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Name: Default Oct 30 00:56:21.190286 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Id: 391b5a6cb3bd4569ad3df5663194d1f4 Oct 30 00:56:21.190286 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Name: tempest-VolumesGetTestJSON-1986195394-project Oct 30 00:56:21.190286 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: method: Oct 30 00:56:21.190286 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: exception: Volume a487f977-045f-450d-b387-d91e31419efe could not be found. Oct 30 00:56:21.190286 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: action_args: {'id': 'a487f977-045f-450d-b387-d91e31419efe'} Oct 30 00:56:21.190286 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 00:56:21.191181 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 00:56:21.191181 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 30 00:56:21.191181 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 30 00:56:21.191181 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 30 00:56:21.191181 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 30 00:56:21.191181 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 30 00:56:21.191181 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 30 00:56:21.191181 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 30 00:56:21.191181 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 30 00:56:21.191181 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 30 00:56:21.191181 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 30 00:56:21.191181 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 30 00:56:21.191181 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 30 00:56:21.191181 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 30 00:56:21.191181 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 30 00:56:21.191181 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume a487f977-045f-450d-b387-d91e31419efe could not be found. (HTTP 404) (Request-ID: req-f8e79e06-bf16-44bc-bf93-38e4508c3387) Oct 30 00:56:21.191181 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 00:56:21.191181 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 00:56:21.191181 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 00:56:21.192651 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 00:56:21.192651 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Oct 30 00:56:21.192651 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Oct 30 00:56:21.192651 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 30 00:56:21.192651 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 30 00:56:21.192651 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 30 00:56:21.192651 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 30 00:56:21.192651 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 30 00:56:21.192651 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 30 00:56:21.192651 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 30 00:56:21.192651 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 30 00:56:21.192651 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 30 00:56:21.192651 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 30 00:56:21.192651 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 30 00:56:21.192651 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 30 00:56:21.192651 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 30 00:56:21.192651 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 30 00:56:21.192651 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 30 00:56:21.192651 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 30 00:56:21.194010 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 30 00:56:21.194010 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 30 00:56:21.194010 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 30 00:56:21.194010 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 30 00:56:21.194010 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume a487f977-045f-450d-b387-d91e31419efe could not be found. Oct 30 00:56:21.194010 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 00:56:21.194010 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 00:56:21.194010 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 00:56:21.194010 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 00:56:21.194010 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 00:56:21.194010 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 00:56:21.194010 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 00:56:21.194010 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 00:56:21.194010 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 30 00:56:21.194010 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 30 00:56:21.194010 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 00:56:21.194010 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 00:56:21.194010 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Oct 30 00:56:21.194010 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 30 00:56:21.195459 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume a487f977-045f-450d-b387-d91e31419efe could not be found. Oct 30 00:56:21.195459 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 00:56:21.195459 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 00:56:21.195459 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 00:56:21.195459 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 00:56:21.195459 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 00:56:21.195459 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 00:56:21.195459 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 00:56:21.195459 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 00:56:21.195459 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume a487f977-045f-450d-b387-d91e31419efe could not be found. Oct 30 00:56:21.195459 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 00:56:21.207854 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-da22c444-7c33-463f-9f89-589d13df12b5 tempest-VolumesGetTestJSON-1986195394 tempest-VolumesGetTestJSON-1986195394-project] Returning 404 to user: Volume a487f977-045f-450d-b387-d91e31419efe could not be found. {{(pid=118962) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 00:56:21.209439 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-da22c444-7c33-463f-9f89-589d13df12b5 tempest-VolumesGetTestJSON-1986195394 tempest-VolumesGetTestJSON-1986195394-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/a487f977-045f-450d-b387-d91e31419efe" status: 404 len: 109 microversion: 2.1 time: 0.146704 Oct 30 00:56:21.213552 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 44/79] 10.222.0.16 () {58 vars in 1245 bytes} [Sun Oct 30 00:56:21 2022] GET /compute/v2.1/os-volumes/a487f977-045f-450d-b387-d91e31419efe => generated 109 bytes in 153 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 30 00:56:21.234402 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-1b738732-2634-4425-81d7-e1726acc12ef tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:21.235758 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-1b738732-2634-4425-81d7-e1726acc12ef tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:21.261169 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1b738732-2634-4425-81d7-e1726acc12ef tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:21.264415 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1b738732-2634-4425-81d7-e1726acc12ef tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:21.265400 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1b738732-2634-4425-81d7-e1726acc12ef tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:21.266378 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1b738732-2634-4425-81d7-e1726acc12ef tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:21.268635 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1b738732-2634-4425-81d7-e1726acc12ef tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:21.269393 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1b738732-2634-4425-81d7-e1726acc12ef tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:21.675064 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-e9e3040e-9476-45d7-af0f-9e4483a67bdc tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c92f7b4d09d74b8d93d739c1938694e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f7d505c7b9e4ca08d6437e6e529a039', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:21.692018 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-e9e3040e-9476-45d7-af0f-9e4483a67bdc tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] 10.222.0.16 "GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1" status: 200 len: 1314 microversion: 2.1 time: 0.606205 Oct 30 00:56:21.704187 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 36/80] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:21 2022] GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1 => generated 1314 bytes in 619 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:21.776153 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-1b738732-2634-4425-81d7-e1726acc12ef tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:21.788968 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-1b738732-2634-4425-81d7-e1726acc12ef tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:21.790618 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-1b738732-2634-4425-81d7-e1726acc12ef tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:21.800792 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-1b738732-2634-4425-81d7-e1726acc12ef tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1406 microversion: 2.20 time: 0.582439 Oct 30 00:56:21.815677 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 45/81] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 00:56:21 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1406 bytes in 600 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 00:56:22.092256 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-c238ae40-f433-4cc6-9098-3ef88a97937a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:22.092941 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-c238ae40-f433-4cc6-9098-3ef88a97937a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:22.104379 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c238ae40-f433-4cc6-9098-3ef88a97937a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:22.106719 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c238ae40-f433-4cc6-9098-3ef88a97937a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:22.107415 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c238ae40-f433-4cc6-9098-3ef88a97937a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:22.109866 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c238ae40-f433-4cc6-9098-3ef88a97937a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:22.111510 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c238ae40-f433-4cc6-9098-3ef88a97937a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:22.112145 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c238ae40-f433-4cc6-9098-3ef88a97937a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:22.112423 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-8bc886a5-35a9-4de5-abe5-48dfe1589e51 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:22.113036 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-8bc886a5-35a9-4de5-abe5-48dfe1589e51 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5652be64-c2a3-4b24-a3aa-c962f204c06e] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:22.123033 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8bc886a5-35a9-4de5-abe5-48dfe1589e51 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:22.124969 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8bc886a5-35a9-4de5-abe5-48dfe1589e51 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:22.125568 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8bc886a5-35a9-4de5-abe5-48dfe1589e51 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:22.127605 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8bc886a5-35a9-4de5-abe5-48dfe1589e51 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:22.145198 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8bc886a5-35a9-4de5-abe5-48dfe1589e51 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.017s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:22.145700 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8bc886a5-35a9-4de5-abe5-48dfe1589e51 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:22.173173 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell1:SELECT=100 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:56:22.300588 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell0:SELECT=2 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:56:22.940455 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-c238ae40-f433-4cc6-9098-3ef88a97937a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:22.958498 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-8bc886a5-35a9-4de5-abe5-48dfe1589e51 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:22.967919 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-8bc886a5-35a9-4de5-abe5-48dfe1589e51 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e" status: 200 len: 1367 microversion: 2.1 time: 0.859270 Oct 30 00:56:22.968863 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-c238ae40-f433-4cc6-9098-3ef88a97937a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1364 microversion: 2.1 time: 0.881481 Oct 30 00:56:22.971277 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 37/82] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:22 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1364 bytes in 885 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:22.985098 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 46/83] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:22 2022] GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e => generated 1367 bytes in 877 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:23.001266 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-d26bdcfe-7724-447c-8440-aabb3a9303c1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:23.003106 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-42ed494f-46ba-4f29-9624-b1ee08c3c2da tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:23.004108 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-42ed494f-46ba-4f29-9624-b1ee08c3c2da tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:23.006110 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-d26bdcfe-7724-447c-8440-aabb3a9303c1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:23.020210 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-42ed494f-46ba-4f29-9624-b1ee08c3c2da tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:23.022512 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-42ed494f-46ba-4f29-9624-b1ee08c3c2da tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:23.023312 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-42ed494f-46ba-4f29-9624-b1ee08c3c2da tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:23.024969 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-42ed494f-46ba-4f29-9624-b1ee08c3c2da tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:23.025866 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d26bdcfe-7724-447c-8440-aabb3a9303c1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:23.026883 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-42ed494f-46ba-4f29-9624-b1ee08c3c2da tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:23.027249 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d26bdcfe-7724-447c-8440-aabb3a9303c1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:23.028788 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-42ed494f-46ba-4f29-9624-b1ee08c3c2da tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:23.028953 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d26bdcfe-7724-447c-8440-aabb3a9303c1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:23.031499 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d26bdcfe-7724-447c-8440-aabb3a9303c1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:23.032207 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d26bdcfe-7724-447c-8440-aabb3a9303c1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:23.033584 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d26bdcfe-7724-447c-8440-aabb3a9303c1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:23.387265 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell0:SELECT=20 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:56:23.453620 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-d26bdcfe-7724-447c-8440-aabb3a9303c1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:23.472705 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-42ed494f-46ba-4f29-9624-b1ee08c3c2da tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c92f7b4d09d74b8d93d739c1938694e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f7d505c7b9e4ca08d6437e6e529a039', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:23.493325 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-42ed494f-46ba-4f29-9624-b1ee08c3c2da tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] 10.222.0.16 "GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1" status: 200 len: 1314 microversion: 2.1 time: 0.520525 Oct 30 00:56:23.497005 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-d26bdcfe-7724-447c-8440-aabb3a9303c1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:23.503012 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 38/84] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:22 2022] GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1 => generated 1314 bytes in 531 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:23.507558 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-d26bdcfe-7724-447c-8440-aabb3a9303c1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:23.514038 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-d26bdcfe-7724-447c-8440-aabb3a9303c1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1406 microversion: 2.20 time: 0.527468 Oct 30 00:56:23.515479 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 47/85] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 00:56:22 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1406 bytes in 530 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 00:56:23.518613 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [req-007b2c3f-ee0b-44ef-977d-b245d480a84a req-149e8662-7661-45e3-ab7b-1a5efde7fb11 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9491a8f4-2b76-4a4e-a6bd-6720114e482f", "tag": "5367850e-73d0-4814-af96-bd430c94c272"}]} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:56:23.543822 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-007b2c3f-ee0b-44ef-977d-b245d480a84a req-149e8662-7661-45e3-ab7b-1a5efde7fb11 service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:23.545140 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-007b2c3f-ee0b-44ef-977d-b245d480a84a req-149e8662-7661-45e3-ab7b-1a5efde7fb11 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:23.545572 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-007b2c3f-ee0b-44ef-977d-b245d480a84a req-149e8662-7661-45e3-ab7b-1a5efde7fb11 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:23.575136 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.compute.server_external_events [req-007b2c3f-ee0b-44ef-977d-b245d480a84a req-149e8662-7661-45e3-ab7b-1a5efde7fb11 service nova] Creating event network-changed:5367850e-73d0-4814-af96-bd430c94c272 for instance 9491a8f4-2b76-4a4e-a6bd-6720114e482f on hv-cinder-maste Oct 30 00:56:23.593700 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [req-007b2c3f-ee0b-44ef-977d-b245d480a84a req-149e8662-7661-45e3-ab7b-1a5efde7fb11 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.086800 Oct 30 00:56:23.595879 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 39/86] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 00:56:23 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:56:23.999663 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-b478e142-9870-4932-82ec-aee9116c5635 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:24.000301 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-b478e142-9870-4932-82ec-aee9116c5635 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:24.002541 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-87af0bd1-7b47-41b9-8974-bb6ad3b15572 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:24.003397 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-87af0bd1-7b47-41b9-8974-bb6ad3b15572 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5652be64-c2a3-4b24-a3aa-c962f204c06e] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:24.006703 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b478e142-9870-4932-82ec-aee9116c5635 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:24.007187 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b478e142-9870-4932-82ec-aee9116c5635 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:24.007828 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b478e142-9870-4932-82ec-aee9116c5635 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:24.008678 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b478e142-9870-4932-82ec-aee9116c5635 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:24.009261 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b478e142-9870-4932-82ec-aee9116c5635 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:24.009829 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b478e142-9870-4932-82ec-aee9116c5635 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:24.010245 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-87af0bd1-7b47-41b9-8974-bb6ad3b15572 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:24.010736 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-87af0bd1-7b47-41b9-8974-bb6ad3b15572 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:24.011173 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-87af0bd1-7b47-41b9-8974-bb6ad3b15572 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:24.011923 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-87af0bd1-7b47-41b9-8974-bb6ad3b15572 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:24.012530 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-87af0bd1-7b47-41b9-8974-bb6ad3b15572 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:24.013247 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-87af0bd1-7b47-41b9-8974-bb6ad3b15572 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:24.228981 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-b478e142-9870-4932-82ec-aee9116c5635 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:24.238815 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-b478e142-9870-4932-82ec-aee9116c5635 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1364 microversion: 2.1 time: 0.244703 Oct 30 00:56:24.240792 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 48/87] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:23 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1364 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:24.270574 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-87af0bd1-7b47-41b9-8974-bb6ad3b15572 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:24.279331 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-87af0bd1-7b47-41b9-8974-bb6ad3b15572 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e" status: 200 len: 1367 microversion: 2.1 time: 0.280909 Oct 30 00:56:24.279940 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 40/88] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:23 2022] GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e => generated 1367 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:24.545636 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-621991c5-d62c-45be-b988-6b9b26ab96df tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:24.546475 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-621991c5-d62c-45be-b988-6b9b26ab96df tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:24.551621 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-a1017662-75ec-48a6-9b99-483a040c99a3 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:24.552392 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-a1017662-75ec-48a6-9b99-483a040c99a3 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:24.553012 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-621991c5-d62c-45be-b988-6b9b26ab96df tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:24.553599 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-621991c5-d62c-45be-b988-6b9b26ab96df tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:24.554124 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-621991c5-d62c-45be-b988-6b9b26ab96df tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:24.554912 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-621991c5-d62c-45be-b988-6b9b26ab96df tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:24.555453 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-621991c5-d62c-45be-b988-6b9b26ab96df tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:24.555987 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-621991c5-d62c-45be-b988-6b9b26ab96df tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:24.559006 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a1017662-75ec-48a6-9b99-483a040c99a3 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:24.559493 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a1017662-75ec-48a6-9b99-483a040c99a3 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:24.559952 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a1017662-75ec-48a6-9b99-483a040c99a3 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:24.562058 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a1017662-75ec-48a6-9b99-483a040c99a3 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:24.562515 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a1017662-75ec-48a6-9b99-483a040c99a3 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:24.563099 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a1017662-75ec-48a6-9b99-483a040c99a3 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:24.760919 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-a1017662-75ec-48a6-9b99-483a040c99a3 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c92f7b4d09d74b8d93d739c1938694e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f7d505c7b9e4ca08d6437e6e529a039', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:24.764012 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-621991c5-d62c-45be-b988-6b9b26ab96df tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:24.770043 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-a1017662-75ec-48a6-9b99-483a040c99a3 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] 10.222.0.16 "GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1" status: 200 len: 1314 microversion: 2.1 time: 0.221667 Oct 30 00:56:24.770043 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 41/89] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:24 2022] GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1 => generated 1314 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 00:56:24.773607 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-621991c5-d62c-45be-b988-6b9b26ab96df tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:24.775222 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-621991c5-d62c-45be-b988-6b9b26ab96df tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:24.776376 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-621991c5-d62c-45be-b988-6b9b26ab96df tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1406 microversion: 2.20 time: 0.234817 Oct 30 00:56:24.776940 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 49/90] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 00:56:24 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1406 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 00:56:25.276648 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-32c5f269-0cb7-4b0e-a777-499b0f24fb17 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:25.277259 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-32c5f269-0cb7-4b0e-a777-499b0f24fb17 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:25.285688 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-32c5f269-0cb7-4b0e-a777-499b0f24fb17 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:25.288402 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-32c5f269-0cb7-4b0e-a777-499b0f24fb17 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:25.288851 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-32c5f269-0cb7-4b0e-a777-499b0f24fb17 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:25.289589 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-32c5f269-0cb7-4b0e-a777-499b0f24fb17 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:25.290045 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-32c5f269-0cb7-4b0e-a777-499b0f24fb17 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:25.290465 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-32c5f269-0cb7-4b0e-a777-499b0f24fb17 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:25.303040 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-84ee9675-1884-4dae-a776-6c188e76affa tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:25.303931 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-84ee9675-1884-4dae-a776-6c188e76affa tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5652be64-c2a3-4b24-a3aa-c962f204c06e] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:25.317096 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-84ee9675-1884-4dae-a776-6c188e76affa tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:25.317908 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-84ee9675-1884-4dae-a776-6c188e76affa tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:25.318618 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-84ee9675-1884-4dae-a776-6c188e76affa tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:25.319512 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-84ee9675-1884-4dae-a776-6c188e76affa tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:25.320478 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-84ee9675-1884-4dae-a776-6c188e76affa tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:25.321100 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-84ee9675-1884-4dae-a776-6c188e76affa tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:25.586361 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-32c5f269-0cb7-4b0e-a777-499b0f24fb17 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:25.597515 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-32c5f269-0cb7-4b0e-a777-499b0f24fb17 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1364 microversion: 2.1 time: 0.336996 Oct 30 00:56:25.598691 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 42/91] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:25 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1364 bytes in 339 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:25.600696 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-84ee9675-1884-4dae-a776-6c188e76affa tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:25.611548 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-84ee9675-1884-4dae-a776-6c188e76affa tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e" status: 200 len: 1367 microversion: 2.1 time: 0.312618 Oct 30 00:56:25.612160 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 50/92] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:25 2022] GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e => generated 1367 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 00:56:25.621113 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [req-b55da3a2-4beb-4899-8362-559073be4a04 req-d1c5ae17-435b-4135-b155-9ba8c11e9676 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "70ad3ec1-64bc-40ac-bb98-cc82d19e2a14", "tag": "dcb64cf9-6d5f-470b-8042-0cdbf3e91991"}]} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:56:25.636087 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-b55da3a2-4beb-4899-8362-559073be4a04 req-d1c5ae17-435b-4135-b155-9ba8c11e9676 service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:25.636600 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-b55da3a2-4beb-4899-8362-559073be4a04 req-d1c5ae17-435b-4135-b155-9ba8c11e9676 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:25.637206 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-b55da3a2-4beb-4899-8362-559073be4a04 req-d1c5ae17-435b-4135-b155-9ba8c11e9676 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:25.653383 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.compute.server_external_events [req-b55da3a2-4beb-4899-8362-559073be4a04 req-d1c5ae17-435b-4135-b155-9ba8c11e9676 service nova] Creating event network-changed:dcb64cf9-6d5f-470b-8042-0cdbf3e91991 for instance 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 on hv-cinder-maste Oct 30 00:56:25.659533 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [req-b55da3a2-4beb-4899-8362-559073be4a04 req-d1c5ae17-435b-4135-b155-9ba8c11e9676 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042199 Oct 30 00:56:25.664032 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 43/93] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 00:56:25 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 30 00:56:25.791856 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-3076938a-c0b1-43a6-954c-1bffcaa4bd1a tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:25.792489 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-3076938a-c0b1-43a6-954c-1bffcaa4bd1a tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:25.801561 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3076938a-c0b1-43a6-954c-1bffcaa4bd1a tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:25.802040 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3076938a-c0b1-43a6-954c-1bffcaa4bd1a tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:25.802272 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-593fcd31-737e-4d19-b1df-3a148d00a563 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:25.802495 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3076938a-c0b1-43a6-954c-1bffcaa4bd1a tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:25.803271 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3076938a-c0b1-43a6-954c-1bffcaa4bd1a tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:25.803473 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-593fcd31-737e-4d19-b1df-3a148d00a563 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:25.803744 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3076938a-c0b1-43a6-954c-1bffcaa4bd1a tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:25.804145 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3076938a-c0b1-43a6-954c-1bffcaa4bd1a tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:25.808429 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell1:SELECT=100 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:56:25.822899 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-593fcd31-737e-4d19-b1df-3a148d00a563 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:25.824716 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-593fcd31-737e-4d19-b1df-3a148d00a563 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:25.825352 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-593fcd31-737e-4d19-b1df-3a148d00a563 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:25.826484 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-593fcd31-737e-4d19-b1df-3a148d00a563 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:25.827152 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-593fcd31-737e-4d19-b1df-3a148d00a563 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:25.827845 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-593fcd31-737e-4d19-b1df-3a148d00a563 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:26.063844 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-593fcd31-737e-4d19-b1df-3a148d00a563 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:26.064636 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-3076938a-c0b1-43a6-954c-1bffcaa4bd1a tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c92f7b4d09d74b8d93d739c1938694e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f7d505c7b9e4ca08d6437e6e529a039', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:26.076171 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-3076938a-c0b1-43a6-954c-1bffcaa4bd1a tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] 10.222.0.16 "GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1" status: 200 len: 1314 microversion: 2.1 time: 0.288137 Oct 30 00:56:26.078554 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 51/94] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:25 2022] GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1 => generated 1314 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:26.079641 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-593fcd31-737e-4d19-b1df-3a148d00a563 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:26.081643 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-593fcd31-737e-4d19-b1df-3a148d00a563 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:26.085647 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-593fcd31-737e-4d19-b1df-3a148d00a563 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1406 microversion: 2.20 time: 0.287888 Oct 30 00:56:26.087080 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 44/95] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 00:56:25 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1406 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 00:56:26.622212 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-9e7fc9d3-53e7-46bd-85ca-bd6d933c5a2a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:26.622866 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-9e7fc9d3-53e7-46bd-85ca-bd6d933c5a2a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:26.631075 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9e7fc9d3-53e7-46bd-85ca-bd6d933c5a2a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:26.631701 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9e7fc9d3-53e7-46bd-85ca-bd6d933c5a2a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:26.632177 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9e7fc9d3-53e7-46bd-85ca-bd6d933c5a2a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:26.632843 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9e7fc9d3-53e7-46bd-85ca-bd6d933c5a2a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:26.633457 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9e7fc9d3-53e7-46bd-85ca-bd6d933c5a2a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:26.633888 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9e7fc9d3-53e7-46bd-85ca-bd6d933c5a2a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:26.634380 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-0a8c398c-6ff0-484a-bf54-b8529c0dfc17 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:26.635483 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-0a8c398c-6ff0-484a-bf54-b8529c0dfc17 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5652be64-c2a3-4b24-a3aa-c962f204c06e] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:26.641768 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0a8c398c-6ff0-484a-bf54-b8529c0dfc17 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:26.642439 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0a8c398c-6ff0-484a-bf54-b8529c0dfc17 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:26.643056 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0a8c398c-6ff0-484a-bf54-b8529c0dfc17 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:26.644013 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0a8c398c-6ff0-484a-bf54-b8529c0dfc17 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:26.644799 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0a8c398c-6ff0-484a-bf54-b8529c0dfc17 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:26.645366 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0a8c398c-6ff0-484a-bf54-b8529c0dfc17 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:26.997358 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-9e7fc9d3-53e7-46bd-85ca-bd6d933c5a2a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:27.001420 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-0a8c398c-6ff0-484a-bf54-b8529c0dfc17 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:27.005466 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-9e7fc9d3-53e7-46bd-85ca-bd6d933c5a2a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1364 microversion: 2.1 time: 0.387169 Oct 30 00:56:27.006049 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 52/96] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:26 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1364 bytes in 388 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:27.009419 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-0a8c398c-6ff0-484a-bf54-b8529c0dfc17 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e" status: 200 len: 1367 microversion: 2.1 time: 0.378773 Oct 30 00:56:27.009984 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 45/97] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:26 2022] GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e => generated 1367 bytes in 380 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:27.106667 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-aa437713-7d2e-4db6-8f0d-d1f6e67fca5c tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:27.107699 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-aa437713-7d2e-4db6-8f0d-d1f6e67fca5c tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:27.116392 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-2cd50120-f492-48fb-a7f7-dc410f81b04d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:27.116932 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-aa437713-7d2e-4db6-8f0d-d1f6e67fca5c tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:27.117082 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-2cd50120-f492-48fb-a7f7-dc410f81b04d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:27.117425 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-aa437713-7d2e-4db6-8f0d-d1f6e67fca5c tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:27.117851 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-aa437713-7d2e-4db6-8f0d-d1f6e67fca5c tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:27.118527 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-aa437713-7d2e-4db6-8f0d-d1f6e67fca5c tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:27.118972 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-aa437713-7d2e-4db6-8f0d-d1f6e67fca5c tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:27.119387 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-aa437713-7d2e-4db6-8f0d-d1f6e67fca5c tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:27.123735 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2cd50120-f492-48fb-a7f7-dc410f81b04d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:27.124362 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2cd50120-f492-48fb-a7f7-dc410f81b04d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:27.124890 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2cd50120-f492-48fb-a7f7-dc410f81b04d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:27.125508 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2cd50120-f492-48fb-a7f7-dc410f81b04d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:27.125958 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2cd50120-f492-48fb-a7f7-dc410f81b04d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:27.126374 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2cd50120-f492-48fb-a7f7-dc410f81b04d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:27.318875 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-aa437713-7d2e-4db6-8f0d-d1f6e67fca5c tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c92f7b4d09d74b8d93d739c1938694e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f7d505c7b9e4ca08d6437e6e529a039', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:27.323627 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-2cd50120-f492-48fb-a7f7-dc410f81b04d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:27.335453 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-aa437713-7d2e-4db6-8f0d-d1f6e67fca5c tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] 10.222.0.16 "GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1" status: 200 len: 1314 microversion: 2.1 time: 0.235476 Oct 30 00:56:27.339008 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-2cd50120-f492-48fb-a7f7-dc410f81b04d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:27.340583 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-2cd50120-f492-48fb-a7f7-dc410f81b04d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:27.341260 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 53/98] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:27 2022] GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1 => generated 1314 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:27.342452 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-2cd50120-f492-48fb-a7f7-dc410f81b04d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1406 microversion: 2.20 time: 0.232673 Oct 30 00:56:27.343389 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 46/99] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 00:56:27 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1406 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 00:56:28.031591 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-d19f8ee4-7f94-4824-8704-8a6ebd252f82 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:28.032677 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-d19f8ee4-7f94-4824-8704-8a6ebd252f82 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:28.036918 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-0d4b9281-8799-4222-a81a-c9211d56e58f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:28.037932 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-0d4b9281-8799-4222-a81a-c9211d56e58f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5652be64-c2a3-4b24-a3aa-c962f204c06e] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:28.042568 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d19f8ee4-7f94-4824-8704-8a6ebd252f82 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:28.043363 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d19f8ee4-7f94-4824-8704-8a6ebd252f82 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:28.044135 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d19f8ee4-7f94-4824-8704-8a6ebd252f82 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:28.045234 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d19f8ee4-7f94-4824-8704-8a6ebd252f82 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:28.045999 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d19f8ee4-7f94-4824-8704-8a6ebd252f82 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:28.046698 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d19f8ee4-7f94-4824-8704-8a6ebd252f82 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:28.047630 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0d4b9281-8799-4222-a81a-c9211d56e58f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:28.048469 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0d4b9281-8799-4222-a81a-c9211d56e58f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:28.049167 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0d4b9281-8799-4222-a81a-c9211d56e58f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:28.050304 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0d4b9281-8799-4222-a81a-c9211d56e58f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:28.051055 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0d4b9281-8799-4222-a81a-c9211d56e58f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:28.051762 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0d4b9281-8799-4222-a81a-c9211d56e58f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:28.240969 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-0d4b9281-8799-4222-a81a-c9211d56e58f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:28.244005 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-d19f8ee4-7f94-4824-8704-8a6ebd252f82 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:28.249307 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-0d4b9281-8799-4222-a81a-c9211d56e58f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e" status: 200 len: 1367 microversion: 2.1 time: 0.218766 Oct 30 00:56:28.249881 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 47/100] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:28 2022] GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e => generated 1367 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:28.251634 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-d19f8ee4-7f94-4824-8704-8a6ebd252f82 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1364 microversion: 2.1 time: 0.226483 Oct 30 00:56:28.252226 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 54/101] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:28 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1364 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:28.362697 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-28f58af8-d380-497b-91f7-0697020f6091 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:28.363484 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-28f58af8-d380-497b-91f7-0697020f6091 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:28.369857 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-28f58af8-d380-497b-91f7-0697020f6091 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:28.370437 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-28f58af8-d380-497b-91f7-0697020f6091 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:28.371070 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-28f58af8-d380-497b-91f7-0697020f6091 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:28.371765 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-a99ee6ed-63c9-4933-8950-f533891899f4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:28.371874 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-28f58af8-d380-497b-91f7-0697020f6091 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:28.372221 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-28f58af8-d380-497b-91f7-0697020f6091 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:28.372388 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-a99ee6ed-63c9-4933-8950-f533891899f4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:28.372617 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-28f58af8-d380-497b-91f7-0697020f6091 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:28.378466 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a99ee6ed-63c9-4933-8950-f533891899f4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:28.378950 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a99ee6ed-63c9-4933-8950-f533891899f4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:28.379402 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a99ee6ed-63c9-4933-8950-f533891899f4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:28.380112 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a99ee6ed-63c9-4933-8950-f533891899f4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:28.380580 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a99ee6ed-63c9-4933-8950-f533891899f4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:28.381088 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a99ee6ed-63c9-4933-8950-f533891899f4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:28.563965 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-28f58af8-d380-497b-91f7-0697020f6091 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c92f7b4d09d74b8d93d739c1938694e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f7d505c7b9e4ca08d6437e6e529a039', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:28.568804 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-a99ee6ed-63c9-4933-8950-f533891899f4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:28.575591 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-28f58af8-d380-497b-91f7-0697020f6091 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] 10.222.0.16 "GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1" status: 200 len: 1314 microversion: 2.1 time: 0.216718 Oct 30 00:56:28.576424 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 48/102] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:28 2022] GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1 => generated 1314 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:28.581111 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-a99ee6ed-63c9-4933-8950-f533891899f4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:28.583066 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-a99ee6ed-63c9-4933-8950-f533891899f4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:28.584634 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-a99ee6ed-63c9-4933-8950-f533891899f4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1406 microversion: 2.20 time: 0.222122 Oct 30 00:56:28.585180 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 55/103] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 00:56:28 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1406 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 00:56:29.274881 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-ba04b5bb-3aaa-4688-a96f-9ef752b4072d tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:29.275446 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-ba04b5bb-3aaa-4688-a96f-9ef752b4072d tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5652be64-c2a3-4b24-a3aa-c962f204c06e] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:29.275700 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-7e70a545-0843-4ec0-bd55-5f70fa2d42a4 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:29.276336 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-7e70a545-0843-4ec0-bd55-5f70fa2d42a4 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:29.277538 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell0:SELECT=8 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:56:29.282956 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7e70a545-0843-4ec0-bd55-5f70fa2d42a4 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:29.283178 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ba04b5bb-3aaa-4688-a96f-9ef752b4072d tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:29.283432 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7e70a545-0843-4ec0-bd55-5f70fa2d42a4 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:29.283642 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ba04b5bb-3aaa-4688-a96f-9ef752b4072d tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:29.283900 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7e70a545-0843-4ec0-bd55-5f70fa2d42a4 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:29.284099 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ba04b5bb-3aaa-4688-a96f-9ef752b4072d tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:29.284576 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7e70a545-0843-4ec0-bd55-5f70fa2d42a4 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:29.284778 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ba04b5bb-3aaa-4688-a96f-9ef752b4072d tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:29.285002 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7e70a545-0843-4ec0-bd55-5f70fa2d42a4 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:29.285224 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ba04b5bb-3aaa-4688-a96f-9ef752b4072d tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:29.285415 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7e70a545-0843-4ec0-bd55-5f70fa2d42a4 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:29.285640 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ba04b5bb-3aaa-4688-a96f-9ef752b4072d tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:29.478758 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-7e70a545-0843-4ec0-bd55-5f70fa2d42a4 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:29.482915 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-ba04b5bb-3aaa-4688-a96f-9ef752b4072d tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:29.487105 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-7e70a545-0843-4ec0-bd55-5f70fa2d42a4 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1364 microversion: 2.1 time: 0.215342 Oct 30 00:56:29.487651 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 56/104] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:29 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1364 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:29.491275 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-ba04b5bb-3aaa-4688-a96f-9ef752b4072d tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e" status: 200 len: 1367 microversion: 2.1 time: 0.220820 Oct 30 00:56:29.491868 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 49/105] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:29 2022] GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e => generated 1367 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:29.602050 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-943eb58e-c843-450c-8c04-0f9e18de2e45 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:29.602658 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-943eb58e-c843-450c-8c04-0f9e18de2e45 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:29.605724 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_api:SELECT=95,nova_api:INSERT=5 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:56:29.609468 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-4c483099-3a21-4db8-a72d-1f9c47f405a9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:29.609607 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-943eb58e-c843-450c-8c04-0f9e18de2e45 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:29.609957 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-943eb58e-c843-450c-8c04-0f9e18de2e45 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:29.610127 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-4c483099-3a21-4db8-a72d-1f9c47f405a9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:29.610373 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-943eb58e-c843-450c-8c04-0f9e18de2e45 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:29.611062 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-943eb58e-c843-450c-8c04-0f9e18de2e45 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:29.611812 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-943eb58e-c843-450c-8c04-0f9e18de2e45 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:29.612230 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-943eb58e-c843-450c-8c04-0f9e18de2e45 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:29.616161 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4c483099-3a21-4db8-a72d-1f9c47f405a9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:29.616637 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4c483099-3a21-4db8-a72d-1f9c47f405a9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:29.617067 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4c483099-3a21-4db8-a72d-1f9c47f405a9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:29.617799 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4c483099-3a21-4db8-a72d-1f9c47f405a9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:29.618260 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4c483099-3a21-4db8-a72d-1f9c47f405a9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:29.618731 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4c483099-3a21-4db8-a72d-1f9c47f405a9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:29.806004 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-943eb58e-c843-450c-8c04-0f9e18de2e45 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c92f7b4d09d74b8d93d739c1938694e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f7d505c7b9e4ca08d6437e6e529a039', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:29.809128 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-4c483099-3a21-4db8-a72d-1f9c47f405a9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:29.814080 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-943eb58e-c843-450c-8c04-0f9e18de2e45 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] 10.222.0.16 "GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1" status: 200 len: 1314 microversion: 2.1 time: 0.215816 Oct 30 00:56:29.814554 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 57/106] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:29 2022] GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1 => generated 1314 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:29.817339 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-4c483099-3a21-4db8-a72d-1f9c47f405a9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:29.818830 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-4c483099-3a21-4db8-a72d-1f9c47f405a9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:29.820560 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-4c483099-3a21-4db8-a72d-1f9c47f405a9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1406 microversion: 2.20 time: 0.215051 Oct 30 00:56:29.821080 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 50/107] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 00:56:29 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1406 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 00:56:30.509896 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-88c7bd31-fa20-47c7-aedc-b79103edbe6c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:30.510510 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-88c7bd31-fa20-47c7-aedc-b79103edbe6c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:30.514533 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-7c8c1d5d-05ec-4293-84a5-f70bc455e238 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:30.515126 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-7c8c1d5d-05ec-4293-84a5-f70bc455e238 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5652be64-c2a3-4b24-a3aa-c962f204c06e] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:30.516808 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-88c7bd31-fa20-47c7-aedc-b79103edbe6c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:30.517313 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-88c7bd31-fa20-47c7-aedc-b79103edbe6c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:30.517749 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-88c7bd31-fa20-47c7-aedc-b79103edbe6c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:30.518448 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-88c7bd31-fa20-47c7-aedc-b79103edbe6c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:30.518893 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-88c7bd31-fa20-47c7-aedc-b79103edbe6c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:30.519310 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-88c7bd31-fa20-47c7-aedc-b79103edbe6c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:30.521253 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7c8c1d5d-05ec-4293-84a5-f70bc455e238 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:30.521729 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7c8c1d5d-05ec-4293-84a5-f70bc455e238 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:30.522156 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7c8c1d5d-05ec-4293-84a5-f70bc455e238 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:30.522848 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7c8c1d5d-05ec-4293-84a5-f70bc455e238 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:30.523292 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7c8c1d5d-05ec-4293-84a5-f70bc455e238 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:30.523707 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7c8c1d5d-05ec-4293-84a5-f70bc455e238 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:30.710731 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-88c7bd31-fa20-47c7-aedc-b79103edbe6c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:30.713529 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-7c8c1d5d-05ec-4293-84a5-f70bc455e238 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:30.719043 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-88c7bd31-fa20-47c7-aedc-b79103edbe6c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1364 microversion: 2.1 time: 0.213201 Oct 30 00:56:30.719586 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 58/108] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:30 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1364 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 00:56:30.721747 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-7c8c1d5d-05ec-4293-84a5-f70bc455e238 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e" status: 200 len: 1367 microversion: 2.1 time: 0.211699 Oct 30 00:56:30.722273 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 51/109] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:30 2022] GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e => generated 1367 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:30.838508 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-510e3f8e-dd37-4967-82a5-e058d97928f9 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:30.839311 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-510e3f8e-dd37-4967-82a5-e058d97928f9 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:30.845327 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-8792370b-018e-4231-8808-08d989e1a480 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:30.845707 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-510e3f8e-dd37-4967-82a5-e058d97928f9 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:30.845942 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-8792370b-018e-4231-8808-08d989e1a480 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:30.846196 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-510e3f8e-dd37-4967-82a5-e058d97928f9 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:30.846630 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-510e3f8e-dd37-4967-82a5-e058d97928f9 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:30.847323 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-510e3f8e-dd37-4967-82a5-e058d97928f9 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:30.847800 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-510e3f8e-dd37-4967-82a5-e058d97928f9 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:30.848223 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-510e3f8e-dd37-4967-82a5-e058d97928f9 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:30.851858 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8792370b-018e-4231-8808-08d989e1a480 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:30.852338 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8792370b-018e-4231-8808-08d989e1a480 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:30.852767 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8792370b-018e-4231-8808-08d989e1a480 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:30.853523 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8792370b-018e-4231-8808-08d989e1a480 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:30.853973 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8792370b-018e-4231-8808-08d989e1a480 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:30.854383 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8792370b-018e-4231-8808-08d989e1a480 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:31.041653 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-510e3f8e-dd37-4967-82a5-e058d97928f9 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c92f7b4d09d74b8d93d739c1938694e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f7d505c7b9e4ca08d6437e6e529a039', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:31.045631 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-8792370b-018e-4231-8808-08d989e1a480 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:31.050062 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-510e3f8e-dd37-4967-82a5-e058d97928f9 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] 10.222.0.16 "GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1" status: 200 len: 1314 microversion: 2.1 time: 0.216395 Oct 30 00:56:31.050887 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 59/110] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:30 2022] GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1 => generated 1314 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:31.055579 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-8792370b-018e-4231-8808-08d989e1a480 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:31.057061 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-8792370b-018e-4231-8808-08d989e1a480 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:31.058182 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-8792370b-018e-4231-8808-08d989e1a480 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1406 microversion: 2.20 time: 0.216680 Oct 30 00:56:31.058867 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 52/111] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 00:56:30 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1406 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 30 00:56:31.745394 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-a37f54aa-16eb-4487-983e-f01a6d004a17 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:31.746048 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-a37f54aa-16eb-4487-983e-f01a6d004a17 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:31.747282 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-2397f408-57c7-4ac2-9cb3-906a83c3711f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:31.748132 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-2397f408-57c7-4ac2-9cb3-906a83c3711f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5652be64-c2a3-4b24-a3aa-c962f204c06e] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:31.754199 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a37f54aa-16eb-4487-983e-f01a6d004a17 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:31.754835 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a37f54aa-16eb-4487-983e-f01a6d004a17 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:31.755401 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a37f54aa-16eb-4487-983e-f01a6d004a17 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:31.756391 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2397f408-57c7-4ac2-9cb3-906a83c3711f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:31.756573 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a37f54aa-16eb-4487-983e-f01a6d004a17 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:31.757029 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2397f408-57c7-4ac2-9cb3-906a83c3711f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:31.757152 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a37f54aa-16eb-4487-983e-f01a6d004a17 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:31.757602 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2397f408-57c7-4ac2-9cb3-906a83c3711f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:31.757732 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a37f54aa-16eb-4487-983e-f01a6d004a17 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:31.758522 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2397f408-57c7-4ac2-9cb3-906a83c3711f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:31.759122 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2397f408-57c7-4ac2-9cb3-906a83c3711f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:31.759679 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2397f408-57c7-4ac2-9cb3-906a83c3711f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:31.952985 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-a37f54aa-16eb-4487-983e-f01a6d004a17 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:31.957268 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-2397f408-57c7-4ac2-9cb3-906a83c3711f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:31.964211 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-a37f54aa-16eb-4487-983e-f01a6d004a17 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1364 microversion: 2.1 time: 0.224093 Oct 30 00:56:31.964906 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 60/112] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:31 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1364 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:31.969130 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-2397f408-57c7-4ac2-9cb3-906a83c3711f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e" status: 200 len: 1367 microversion: 2.1 time: 0.226685 Oct 30 00:56:31.969857 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 53/113] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:31 2022] GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e => generated 1367 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:32.077620 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-76a278ac-0772-486b-b632-7da6b4dad740 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:32.078832 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-76a278ac-0772-486b-b632-7da6b4dad740 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:32.086969 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-21d74ca5-303a-4ef4-8d87-4881ebc58dc8 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:32.088195 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-21d74ca5-303a-4ef4-8d87-4881ebc58dc8 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:32.089116 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-76a278ac-0772-486b-b632-7da6b4dad740 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:32.089914 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-76a278ac-0772-486b-b632-7da6b4dad740 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:32.090675 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-76a278ac-0772-486b-b632-7da6b4dad740 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:32.091803 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-76a278ac-0772-486b-b632-7da6b4dad740 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:32.092586 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-76a278ac-0772-486b-b632-7da6b4dad740 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:32.093276 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-76a278ac-0772-486b-b632-7da6b4dad740 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:32.099425 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-21d74ca5-303a-4ef4-8d87-4881ebc58dc8 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:32.100211 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-21d74ca5-303a-4ef4-8d87-4881ebc58dc8 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:32.100664 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-21d74ca5-303a-4ef4-8d87-4881ebc58dc8 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:32.101359 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-21d74ca5-303a-4ef4-8d87-4881ebc58dc8 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:32.101810 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-21d74ca5-303a-4ef4-8d87-4881ebc58dc8 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:32.102223 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-21d74ca5-303a-4ef4-8d87-4881ebc58dc8 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:32.286806 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-76a278ac-0772-486b-b632-7da6b4dad740 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c92f7b4d09d74b8d93d739c1938694e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f7d505c7b9e4ca08d6437e6e529a039', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:32.290011 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-21d74ca5-303a-4ef4-8d87-4881ebc58dc8 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:32.295421 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-76a278ac-0772-486b-b632-7da6b4dad740 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] 10.222.0.16 "GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1" status: 200 len: 1314 microversion: 2.1 time: 0.224122 Oct 30 00:56:32.296176 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 61/114] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:32 2022] GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1 => generated 1314 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:32.298357 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-21d74ca5-303a-4ef4-8d87-4881ebc58dc8 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:32.299828 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-21d74ca5-303a-4ef4-8d87-4881ebc58dc8 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:32.300935 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-21d74ca5-303a-4ef4-8d87-4881ebc58dc8 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1406 microversion: 2.20 time: 0.220200 Oct 30 00:56:32.301500 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 54/115] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 00:56:32 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1406 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 00:56:32.990251 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-18b17693-058b-43d7-aed6-0f55f36e7a21 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:32.991028 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-18b17693-058b-43d7-aed6-0f55f36e7a21 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:32.995009 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-46b65f0e-5c8a-4447-8694-f9ee282e40a0 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:32.995655 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-46b65f0e-5c8a-4447-8694-f9ee282e40a0 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5652be64-c2a3-4b24-a3aa-c962f204c06e] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:32.997057 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-18b17693-058b-43d7-aed6-0f55f36e7a21 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:32.997537 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-18b17693-058b-43d7-aed6-0f55f36e7a21 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:32.997955 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-18b17693-058b-43d7-aed6-0f55f36e7a21 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:32.998671 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-18b17693-058b-43d7-aed6-0f55f36e7a21 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:32.999119 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-18b17693-058b-43d7-aed6-0f55f36e7a21 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:32.999531 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-18b17693-058b-43d7-aed6-0f55f36e7a21 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:33.001674 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-46b65f0e-5c8a-4447-8694-f9ee282e40a0 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:33.002155 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-46b65f0e-5c8a-4447-8694-f9ee282e40a0 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:33.002564 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-46b65f0e-5c8a-4447-8694-f9ee282e40a0 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:33.003261 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-46b65f0e-5c8a-4447-8694-f9ee282e40a0 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:33.003704 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-46b65f0e-5c8a-4447-8694-f9ee282e40a0 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:33.004121 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-46b65f0e-5c8a-4447-8694-f9ee282e40a0 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:33.021987 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell1:SELECT=100 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:56:33.195983 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-18b17693-058b-43d7-aed6-0f55f36e7a21 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:33.199182 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-46b65f0e-5c8a-4447-8694-f9ee282e40a0 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:33.203993 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-18b17693-058b-43d7-aed6-0f55f36e7a21 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1364 microversion: 2.1 time: 0.217568 Oct 30 00:56:33.204563 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 62/116] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:32 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1364 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:33.207092 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-46b65f0e-5c8a-4447-8694-f9ee282e40a0 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e" status: 200 len: 1367 microversion: 2.1 time: 0.216421 Oct 30 00:56:33.207684 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 55/117] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:32 2022] GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e => generated 1367 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:33.321464 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-2032a41d-93d4-49de-bc51-627d93f7e8c5 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:33.322075 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-2032a41d-93d4-49de-bc51-627d93f7e8c5 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:33.326030 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-51014397-0416-4d41-b339-dced4902c92f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:33.326635 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-51014397-0416-4d41-b339-dced4902c92f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:33.328453 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2032a41d-93d4-49de-bc51-627d93f7e8c5 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:33.328937 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2032a41d-93d4-49de-bc51-627d93f7e8c5 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:33.329365 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2032a41d-93d4-49de-bc51-627d93f7e8c5 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:33.330058 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2032a41d-93d4-49de-bc51-627d93f7e8c5 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:33.330505 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2032a41d-93d4-49de-bc51-627d93f7e8c5 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:33.330923 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2032a41d-93d4-49de-bc51-627d93f7e8c5 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:33.332576 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-51014397-0416-4d41-b339-dced4902c92f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:33.333693 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-51014397-0416-4d41-b339-dced4902c92f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:33.334104 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-51014397-0416-4d41-b339-dced4902c92f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:33.334815 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-51014397-0416-4d41-b339-dced4902c92f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:33.335261 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-51014397-0416-4d41-b339-dced4902c92f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:33.335683 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-51014397-0416-4d41-b339-dced4902c92f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:33.681152 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-2032a41d-93d4-49de-bc51-627d93f7e8c5 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c92f7b4d09d74b8d93d739c1938694e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f7d505c7b9e4ca08d6437e6e529a039', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:33.684852 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-51014397-0416-4d41-b339-dced4902c92f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:33.689363 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-2032a41d-93d4-49de-bc51-627d93f7e8c5 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] 10.222.0.16 "GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1" status: 200 len: 1314 microversion: 2.1 time: 0.373179 Oct 30 00:56:33.690095 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 63/118] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:33 2022] GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1 => generated 1314 bytes in 375 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:33.692316 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-51014397-0416-4d41-b339-dced4902c92f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:33.693828 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-51014397-0416-4d41-b339-dced4902c92f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:33.695311 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-51014397-0416-4d41-b339-dced4902c92f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1406 microversion: 2.20 time: 0.373416 Oct 30 00:56:33.696150 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 56/119] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 00:56:33 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1406 bytes in 375 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 00:56:34.229368 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-eba20622-775f-4f84-868b-e33e85f76c2f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:34.230032 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-eba20622-775f-4f84-868b-e33e85f76c2f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:34.231168 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-66374cfe-9073-43b8-931b-920ca7c3a123 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:34.231988 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-66374cfe-9073-43b8-931b-920ca7c3a123 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5652be64-c2a3-4b24-a3aa-c962f204c06e] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:34.237663 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-eba20622-775f-4f84-868b-e33e85f76c2f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:34.238267 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-eba20622-775f-4f84-868b-e33e85f76c2f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:34.238809 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-eba20622-775f-4f84-868b-e33e85f76c2f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:34.239594 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-66374cfe-9073-43b8-931b-920ca7c3a123 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:34.239794 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-eba20622-775f-4f84-868b-e33e85f76c2f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:34.240222 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-eba20622-775f-4f84-868b-e33e85f76c2f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:34.240382 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-66374cfe-9073-43b8-931b-920ca7c3a123 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:34.240743 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-eba20622-775f-4f84-868b-e33e85f76c2f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:34.240866 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-66374cfe-9073-43b8-931b-920ca7c3a123 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:34.241607 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-66374cfe-9073-43b8-931b-920ca7c3a123 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:34.242168 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-66374cfe-9073-43b8-931b-920ca7c3a123 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:34.242703 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-66374cfe-9073-43b8-931b-920ca7c3a123 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:34.436128 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-eba20622-775f-4f84-868b-e33e85f76c2f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:34.443510 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-66374cfe-9073-43b8-931b-920ca7c3a123 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:34.444149 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-eba20622-775f-4f84-868b-e33e85f76c2f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1364 microversion: 2.1 time: 0.220204 Oct 30 00:56:34.445042 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 64/120] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:34 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1364 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:34.452178 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-66374cfe-9073-43b8-931b-920ca7c3a123 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e" status: 200 len: 1367 microversion: 2.1 time: 0.225751 Oct 30 00:56:34.452742 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 57/121] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:34 2022] GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e => generated 1367 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:34.716726 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-98aab330-86b0-493f-8eea-a8658f880d47 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:34.717379 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-98aab330-86b0-493f-8eea-a8658f880d47 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:34.719974 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-2b8fd158-af64-4016-b352-d31bb3cf3bcc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:34.720954 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-2b8fd158-af64-4016-b352-d31bb3cf3bcc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:34.723761 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-98aab330-86b0-493f-8eea-a8658f880d47 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:34.724269 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-98aab330-86b0-493f-8eea-a8658f880d47 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:34.724733 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-98aab330-86b0-493f-8eea-a8658f880d47 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:34.725502 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-98aab330-86b0-493f-8eea-a8658f880d47 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:34.726036 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-98aab330-86b0-493f-8eea-a8658f880d47 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:34.726376 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-98aab330-86b0-493f-8eea-a8658f880d47 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:34.727818 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2b8fd158-af64-4016-b352-d31bb3cf3bcc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:34.728297 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2b8fd158-af64-4016-b352-d31bb3cf3bcc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:34.728805 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2b8fd158-af64-4016-b352-d31bb3cf3bcc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:34.729538 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2b8fd158-af64-4016-b352-d31bb3cf3bcc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:34.730033 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2b8fd158-af64-4016-b352-d31bb3cf3bcc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:34.730702 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2b8fd158-af64-4016-b352-d31bb3cf3bcc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:34.929173 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-98aab330-86b0-493f-8eea-a8658f880d47 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c92f7b4d09d74b8d93d739c1938694e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f7d505c7b9e4ca08d6437e6e529a039', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:34.933034 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-2b8fd158-af64-4016-b352-d31bb3cf3bcc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:34.937708 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-98aab330-86b0-493f-8eea-a8658f880d47 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] 10.222.0.16 "GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1" status: 200 len: 1314 microversion: 2.1 time: 0.225039 Oct 30 00:56:34.938275 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 65/122] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:34 2022] GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1 => generated 1314 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:34.941246 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-2b8fd158-af64-4016-b352-d31bb3cf3bcc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:34.942734 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-2b8fd158-af64-4016-b352-d31bb3cf3bcc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:34.943984 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-2b8fd158-af64-4016-b352-d31bb3cf3bcc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1406 microversion: 2.20 time: 0.228103 Oct 30 00:56:34.944537 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 58/123] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 00:56:34 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1406 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 00:56:35.469523 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-3d0bdb96-fbff-424b-b245-9d7e285ed954 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:35.470198 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-3d0bdb96-fbff-424b-b245-9d7e285ed954 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:35.474882 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-551ec126-4bbb-4c33-b6c7-8d2178b20940 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:35.475559 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-551ec126-4bbb-4c33-b6c7-8d2178b20940 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5652be64-c2a3-4b24-a3aa-c962f204c06e] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:35.477164 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3d0bdb96-fbff-424b-b245-9d7e285ed954 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:35.477698 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3d0bdb96-fbff-424b-b245-9d7e285ed954 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:35.478169 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3d0bdb96-fbff-424b-b245-9d7e285ed954 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:35.478923 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3d0bdb96-fbff-424b-b245-9d7e285ed954 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:35.479412 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3d0bdb96-fbff-424b-b245-9d7e285ed954 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:35.479912 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3d0bdb96-fbff-424b-b245-9d7e285ed954 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:35.482226 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-551ec126-4bbb-4c33-b6c7-8d2178b20940 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:35.482755 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-551ec126-4bbb-4c33-b6c7-8d2178b20940 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:35.483263 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-551ec126-4bbb-4c33-b6c7-8d2178b20940 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:35.484041 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-551ec126-4bbb-4c33-b6c7-8d2178b20940 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:35.484571 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-551ec126-4bbb-4c33-b6c7-8d2178b20940 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:35.485017 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-551ec126-4bbb-4c33-b6c7-8d2178b20940 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:35.672213 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-3d0bdb96-fbff-424b-b245-9d7e285ed954 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:35.675968 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-551ec126-4bbb-4c33-b6c7-8d2178b20940 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:35.680459 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-3d0bdb96-fbff-424b-b245-9d7e285ed954 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1364 microversion: 2.1 time: 0.216279 Oct 30 00:56:35.683848 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 66/124] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:35 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1364 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:35.683993 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-551ec126-4bbb-4c33-b6c7-8d2178b20940 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e" status: 200 len: 1367 microversion: 2.1 time: 0.213009 Oct 30 00:56:35.684299 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 59/125] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:35 2022] GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e => generated 1367 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:35.962120 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-9faf7aee-1b02-4c7f-bb13-644d01081fc6 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:35.964346 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-9faf7aee-1b02-4c7f-bb13-644d01081fc6 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:35.969318 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-81b9009b-0899-4a75-a11e-78206e48d382 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:35.969927 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-81b9009b-0899-4a75-a11e-78206e48d382 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:35.970673 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9faf7aee-1b02-4c7f-bb13-644d01081fc6 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:35.971161 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9faf7aee-1b02-4c7f-bb13-644d01081fc6 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:35.971582 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9faf7aee-1b02-4c7f-bb13-644d01081fc6 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:35.972321 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9faf7aee-1b02-4c7f-bb13-644d01081fc6 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:35.972760 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9faf7aee-1b02-4c7f-bb13-644d01081fc6 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:35.973174 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9faf7aee-1b02-4c7f-bb13-644d01081fc6 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:35.976591 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-81b9009b-0899-4a75-a11e-78206e48d382 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:35.977167 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-81b9009b-0899-4a75-a11e-78206e48d382 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:35.977767 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-81b9009b-0899-4a75-a11e-78206e48d382 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:35.978618 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-81b9009b-0899-4a75-a11e-78206e48d382 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:35.979330 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-81b9009b-0899-4a75-a11e-78206e48d382 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:35.979910 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-81b9009b-0899-4a75-a11e-78206e48d382 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:36.192060 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-9faf7aee-1b02-4c7f-bb13-644d01081fc6 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c92f7b4d09d74b8d93d739c1938694e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f7d505c7b9e4ca08d6437e6e529a039', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:36.196291 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell1:SELECT=100 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:56:36.206024 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-81b9009b-0899-4a75-a11e-78206e48d382 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:36.211169 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-9faf7aee-1b02-4c7f-bb13-644d01081fc6 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] 10.222.0.16 "GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1" status: 200 len: 1314 microversion: 2.1 time: 0.253719 Oct 30 00:56:36.211846 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 67/126] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:35 2022] GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1 => generated 1314 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:36.219659 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-81b9009b-0899-4a75-a11e-78206e48d382 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:36.223949 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-81b9009b-0899-4a75-a11e-78206e48d382 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:36.223949 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-81b9009b-0899-4a75-a11e-78206e48d382 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1406 microversion: 2.20 time: 0.257314 Oct 30 00:56:36.223949 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 60/127] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 00:56:35 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1406 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 00:56:36.703738 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-72fc0932-77e6-427c-acbe-1390e61bbee4 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:36.704947 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-72fc0932-77e6-427c-acbe-1390e61bbee4 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:36.706922 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-c1b018bb-478d-4a03-b1fc-3b1310950e0b tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:36.707530 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-c1b018bb-478d-4a03-b1fc-3b1310950e0b tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5652be64-c2a3-4b24-a3aa-c962f204c06e] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:36.711121 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-72fc0932-77e6-427c-acbe-1390e61bbee4 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:36.711778 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-72fc0932-77e6-427c-acbe-1390e61bbee4 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:36.712359 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-72fc0932-77e6-427c-acbe-1390e61bbee4 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:36.713261 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-72fc0932-77e6-427c-acbe-1390e61bbee4 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:36.714006 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-72fc0932-77e6-427c-acbe-1390e61bbee4 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:36.714680 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-72fc0932-77e6-427c-acbe-1390e61bbee4 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:36.715224 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c1b018bb-478d-4a03-b1fc-3b1310950e0b tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:36.715705 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c1b018bb-478d-4a03-b1fc-3b1310950e0b tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:36.716325 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c1b018bb-478d-4a03-b1fc-3b1310950e0b tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:36.717227 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c1b018bb-478d-4a03-b1fc-3b1310950e0b tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:36.718087 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c1b018bb-478d-4a03-b1fc-3b1310950e0b tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:36.718701 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c1b018bb-478d-4a03-b1fc-3b1310950e0b tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:36.936081 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-72fc0932-77e6-427c-acbe-1390e61bbee4 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:36.940941 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-c1b018bb-478d-4a03-b1fc-3b1310950e0b tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:36.950138 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-72fc0932-77e6-427c-acbe-1390e61bbee4 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1364 microversion: 2.1 time: 0.249990 Oct 30 00:56:36.950321 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-c1b018bb-478d-4a03-b1fc-3b1310950e0b tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e" status: 200 len: 1367 microversion: 2.1 time: 0.247564 Oct 30 00:56:36.950845 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 68/128] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:36 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1364 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:36.951399 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 61/129] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:36 2022] GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e => generated 1367 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:37.239935 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-0bbccfcc-2ec1-463e-82f6-fcda83297797 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:37.239935 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-0bbccfcc-2ec1-463e-82f6-fcda83297797 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:37.243219 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0bbccfcc-2ec1-463e-82f6-fcda83297797 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:37.244103 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0bbccfcc-2ec1-463e-82f6-fcda83297797 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:37.244744 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0bbccfcc-2ec1-463e-82f6-fcda83297797 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:37.245579 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0bbccfcc-2ec1-463e-82f6-fcda83297797 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:37.246544 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0bbccfcc-2ec1-463e-82f6-fcda83297797 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:37.247160 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0bbccfcc-2ec1-463e-82f6-fcda83297797 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:37.248967 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-f3974b38-f231-46de-9e08-2618552835ad tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:37.250193 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-f3974b38-f231-46de-9e08-2618552835ad tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:37.258572 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f3974b38-f231-46de-9e08-2618552835ad tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:37.259075 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f3974b38-f231-46de-9e08-2618552835ad tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:37.259525 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f3974b38-f231-46de-9e08-2618552835ad tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:37.261036 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f3974b38-f231-46de-9e08-2618552835ad tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:37.261503 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f3974b38-f231-46de-9e08-2618552835ad tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:37.261924 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f3974b38-f231-46de-9e08-2618552835ad tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:37.511469 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-0bbccfcc-2ec1-463e-82f6-fcda83297797 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c92f7b4d09d74b8d93d739c1938694e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f7d505c7b9e4ca08d6437e6e529a039', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:37.521786 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-f3974b38-f231-46de-9e08-2618552835ad tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:37.523446 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-0bbccfcc-2ec1-463e-82f6-fcda83297797 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] 10.222.0.16 "GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1" status: 200 len: 1314 microversion: 2.1 time: 0.293823 Oct 30 00:56:37.525024 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 69/130] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:37 2022] GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1 => generated 1314 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:37.539201 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-f3974b38-f231-46de-9e08-2618552835ad tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:37.540853 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-f3974b38-f231-46de-9e08-2618552835ad tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:37.549255 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-f3974b38-f231-46de-9e08-2618552835ad tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1406 microversion: 2.20 time: 0.304132 Oct 30 00:56:37.551235 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 62/131] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 00:56:37 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1406 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 30 00:56:37.977322 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-b2ea5cbe-263e-4d2d-823f-4fa6f5bda15d tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:37.978324 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-b2ea5cbe-263e-4d2d-823f-4fa6f5bda15d tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:37.979178 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-5040c498-eb13-49aa-a342-c668ab8ecc98 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:37.980113 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-5040c498-eb13-49aa-a342-c668ab8ecc98 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5652be64-c2a3-4b24-a3aa-c962f204c06e] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:37.985527 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b2ea5cbe-263e-4d2d-823f-4fa6f5bda15d tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:37.986109 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b2ea5cbe-263e-4d2d-823f-4fa6f5bda15d tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:37.986636 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b2ea5cbe-263e-4d2d-823f-4fa6f5bda15d tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:37.987453 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b2ea5cbe-263e-4d2d-823f-4fa6f5bda15d tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:37.987556 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5040c498-eb13-49aa-a342-c668ab8ecc98 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:37.988332 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b2ea5cbe-263e-4d2d-823f-4fa6f5bda15d tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:37.988839 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5040c498-eb13-49aa-a342-c668ab8ecc98 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:37.988933 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b2ea5cbe-263e-4d2d-823f-4fa6f5bda15d tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:37.989626 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5040c498-eb13-49aa-a342-c668ab8ecc98 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:37.990878 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5040c498-eb13-49aa-a342-c668ab8ecc98 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:37.991327 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5040c498-eb13-49aa-a342-c668ab8ecc98 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:37.991994 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5040c498-eb13-49aa-a342-c668ab8ecc98 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:38.186677 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-5040c498-eb13-49aa-a342-c668ab8ecc98 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:38.201814 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-b2ea5cbe-263e-4d2d-823f-4fa6f5bda15d tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:38.202049 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-5040c498-eb13-49aa-a342-c668ab8ecc98 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e" status: 200 len: 1367 microversion: 2.1 time: 0.225039 Oct 30 00:56:38.202139 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-b2ea5cbe-263e-4d2d-823f-4fa6f5bda15d tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1364 microversion: 2.1 time: 0.228727 Oct 30 00:56:38.202560 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 63/132] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:37 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1364 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:38.206741 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 70/133] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:37 2022] GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e => generated 1367 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:38.553276 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-0cb6b092-812f-4c7f-a5d9-06df25c8f53f tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:38.554638 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-0cb6b092-812f-4c7f-a5d9-06df25c8f53f tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:38.565217 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0cb6b092-812f-4c7f-a5d9-06df25c8f53f tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:38.566063 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0cb6b092-812f-4c7f-a5d9-06df25c8f53f tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:38.566756 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0cb6b092-812f-4c7f-a5d9-06df25c8f53f tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:38.568131 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0cb6b092-812f-4c7f-a5d9-06df25c8f53f tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:38.568880 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0cb6b092-812f-4c7f-a5d9-06df25c8f53f tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:38.569641 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0cb6b092-812f-4c7f-a5d9-06df25c8f53f tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:38.579485 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-e9bbae5c-2b1c-4579-b06e-dfb9960d3231 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:38.580137 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-e9bbae5c-2b1c-4579-b06e-dfb9960d3231 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:38.587440 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e9bbae5c-2b1c-4579-b06e-dfb9960d3231 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:38.587935 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e9bbae5c-2b1c-4579-b06e-dfb9960d3231 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:38.588509 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e9bbae5c-2b1c-4579-b06e-dfb9960d3231 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:38.589350 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e9bbae5c-2b1c-4579-b06e-dfb9960d3231 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:38.589771 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e9bbae5c-2b1c-4579-b06e-dfb9960d3231 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:38.590198 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e9bbae5c-2b1c-4579-b06e-dfb9960d3231 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:38.766826 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-0cb6b092-812f-4c7f-a5d9-06df25c8f53f tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c92f7b4d09d74b8d93d739c1938694e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f7d505c7b9e4ca08d6437e6e529a039', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:38.774512 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-0cb6b092-812f-4c7f-a5d9-06df25c8f53f tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] 10.222.0.16 "GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1" status: 200 len: 1314 microversion: 2.1 time: 0.227732 Oct 30 00:56:38.775088 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 64/134] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:38 2022] GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1 => generated 1314 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:38.778066 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-e9bbae5c-2b1c-4579-b06e-dfb9960d3231 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:38.786260 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-e9bbae5c-2b1c-4579-b06e-dfb9960d3231 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:38.787673 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-e9bbae5c-2b1c-4579-b06e-dfb9960d3231 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:38.788879 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-e9bbae5c-2b1c-4579-b06e-dfb9960d3231 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1406 microversion: 2.20 time: 0.213211 Oct 30 00:56:38.789726 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 71/135] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 00:56:38 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1406 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 00:56:39.219796 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-560ef33e-3256-4ed6-a7c1-5a7e5d8c8e92 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:39.219796 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-560ef33e-3256-4ed6-a7c1-5a7e5d8c8e92 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5652be64-c2a3-4b24-a3aa-c962f204c06e] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:39.224972 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-562ea8fd-3ef6-4617-a5b6-7f85c3cad7cf tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:39.225801 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-560ef33e-3256-4ed6-a7c1-5a7e5d8c8e92 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:39.226466 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-560ef33e-3256-4ed6-a7c1-5a7e5d8c8e92 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:39.227222 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-560ef33e-3256-4ed6-a7c1-5a7e5d8c8e92 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:39.227580 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-560ef33e-3256-4ed6-a7c1-5a7e5d8c8e92 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:39.227971 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-562ea8fd-3ef6-4617-a5b6-7f85c3cad7cf tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:39.228072 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-560ef33e-3256-4ed6-a7c1-5a7e5d8c8e92 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:39.228458 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-560ef33e-3256-4ed6-a7c1-5a7e5d8c8e92 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:39.234875 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-562ea8fd-3ef6-4617-a5b6-7f85c3cad7cf tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:39.236813 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-562ea8fd-3ef6-4617-a5b6-7f85c3cad7cf tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:39.238261 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-562ea8fd-3ef6-4617-a5b6-7f85c3cad7cf tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:39.239589 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-562ea8fd-3ef6-4617-a5b6-7f85c3cad7cf tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:39.240139 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-562ea8fd-3ef6-4617-a5b6-7f85c3cad7cf tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:39.240739 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-562ea8fd-3ef6-4617-a5b6-7f85c3cad7cf tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:39.434062 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-560ef33e-3256-4ed6-a7c1-5a7e5d8c8e92 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:39.441111 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-562ea8fd-3ef6-4617-a5b6-7f85c3cad7cf tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:39.442143 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-560ef33e-3256-4ed6-a7c1-5a7e5d8c8e92 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e" status: 200 len: 1367 microversion: 2.1 time: 0.227183 Oct 30 00:56:39.442717 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 65/136] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:39 2022] GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e => generated 1367 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:39.449990 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-562ea8fd-3ef6-4617-a5b6-7f85c3cad7cf tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1364 microversion: 2.1 time: 0.228839 Oct 30 00:56:39.450551 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 72/137] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:39 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1364 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:39.797738 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-c24c41a7-8884-43f2-9020-0c1913060a15 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:39.798350 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-c24c41a7-8884-43f2-9020-0c1913060a15 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:39.806130 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c24c41a7-8884-43f2-9020-0c1913060a15 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:39.806610 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c24c41a7-8884-43f2-9020-0c1913060a15 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:39.807035 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c24c41a7-8884-43f2-9020-0c1913060a15 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:39.807753 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c24c41a7-8884-43f2-9020-0c1913060a15 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:39.808214 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c24c41a7-8884-43f2-9020-0c1913060a15 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:39.808631 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c24c41a7-8884-43f2-9020-0c1913060a15 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:39.824678 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-3ba58380-4d5e-4b18-a9fa-0f99a8aa82e9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:39.825516 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-3ba58380-4d5e-4b18-a9fa-0f99a8aa82e9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:39.832936 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3ba58380-4d5e-4b18-a9fa-0f99a8aa82e9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:39.833533 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3ba58380-4d5e-4b18-a9fa-0f99a8aa82e9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:39.834095 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3ba58380-4d5e-4b18-a9fa-0f99a8aa82e9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:39.835030 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3ba58380-4d5e-4b18-a9fa-0f99a8aa82e9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:39.835711 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3ba58380-4d5e-4b18-a9fa-0f99a8aa82e9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:39.836159 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3ba58380-4d5e-4b18-a9fa-0f99a8aa82e9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:40.012829 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-c24c41a7-8884-43f2-9020-0c1913060a15 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c92f7b4d09d74b8d93d739c1938694e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f7d505c7b9e4ca08d6437e6e529a039', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:40.017665 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-3ba58380-4d5e-4b18-a9fa-0f99a8aa82e9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:40.026385 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-c24c41a7-8884-43f2-9020-0c1913060a15 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] 10.222.0.16 "GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1" status: 200 len: 1314 microversion: 2.1 time: 0.232810 Oct 30 00:56:40.027101 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 66/138] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:39 2022] GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1 => generated 1314 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:40.027752 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-3ba58380-4d5e-4b18-a9fa-0f99a8aa82e9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:40.029186 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-3ba58380-4d5e-4b18-a9fa-0f99a8aa82e9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:40.030293 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-3ba58380-4d5e-4b18-a9fa-0f99a8aa82e9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1406 microversion: 2.20 time: 0.210912 Oct 30 00:56:40.030854 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 73/139] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 00:56:39 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1406 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 00:56:40.470036 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-c3233a66-cbb7-4402-8d84-73c9bf310030 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:40.471035 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-c3233a66-cbb7-4402-8d84-73c9bf310030 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5652be64-c2a3-4b24-a3aa-c962f204c06e] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:40.476742 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-9da5b483-5d8e-4812-b24e-54ec383f5ba8 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:40.477752 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-9da5b483-5d8e-4812-b24e-54ec383f5ba8 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:40.481293 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c3233a66-cbb7-4402-8d84-73c9bf310030 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:40.482064 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c3233a66-cbb7-4402-8d84-73c9bf310030 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:40.482813 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c3233a66-cbb7-4402-8d84-73c9bf310030 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:40.484041 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c3233a66-cbb7-4402-8d84-73c9bf310030 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:40.484782 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c3233a66-cbb7-4402-8d84-73c9bf310030 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:40.485479 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c3233a66-cbb7-4402-8d84-73c9bf310030 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:40.487661 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9da5b483-5d8e-4812-b24e-54ec383f5ba8 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:40.488621 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9da5b483-5d8e-4812-b24e-54ec383f5ba8 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:40.489303 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9da5b483-5d8e-4812-b24e-54ec383f5ba8 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:40.490433 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9da5b483-5d8e-4812-b24e-54ec383f5ba8 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:40.491226 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9da5b483-5d8e-4812-b24e-54ec383f5ba8 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:40.491847 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9da5b483-5d8e-4812-b24e-54ec383f5ba8 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:40.675465 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-c3233a66-cbb7-4402-8d84-73c9bf310030 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:40.682436 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-9da5b483-5d8e-4812-b24e-54ec383f5ba8 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:40.683763 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-c3233a66-cbb7-4402-8d84-73c9bf310030 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e" status: 200 len: 1367 microversion: 2.1 time: 0.220234 Oct 30 00:56:40.684487 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 67/140] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:40 2022] GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e => generated 1367 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:40.690458 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-9da5b483-5d8e-4812-b24e-54ec383f5ba8 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1364 microversion: 2.1 time: 0.220090 Oct 30 00:56:40.690991 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 74/141] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:40 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1364 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:41.053467 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-d82c0b02-48a4-4a60-aa61-d87a63c95955 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:41.054976 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-d82c0b02-48a4-4a60-aa61-d87a63c95955 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:41.058542 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-ad2cf33c-b292-44fd-8ea1-28ac457d0a18 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:41.059547 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-ad2cf33c-b292-44fd-8ea1-28ac457d0a18 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:41.064988 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d82c0b02-48a4-4a60-aa61-d87a63c95955 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:41.065772 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d82c0b02-48a4-4a60-aa61-d87a63c95955 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:41.066514 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d82c0b02-48a4-4a60-aa61-d87a63c95955 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:41.067590 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d82c0b02-48a4-4a60-aa61-d87a63c95955 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:41.068406 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d82c0b02-48a4-4a60-aa61-d87a63c95955 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:41.068583 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ad2cf33c-b292-44fd-8ea1-28ac457d0a18 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:41.069309 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d82c0b02-48a4-4a60-aa61-d87a63c95955 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:41.069513 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ad2cf33c-b292-44fd-8ea1-28ac457d0a18 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:41.070322 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ad2cf33c-b292-44fd-8ea1-28ac457d0a18 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:41.071019 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ad2cf33c-b292-44fd-8ea1-28ac457d0a18 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:41.071482 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ad2cf33c-b292-44fd-8ea1-28ac457d0a18 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:41.071929 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ad2cf33c-b292-44fd-8ea1-28ac457d0a18 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:41.264247 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-d82c0b02-48a4-4a60-aa61-d87a63c95955 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c92f7b4d09d74b8d93d739c1938694e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f7d505c7b9e4ca08d6437e6e529a039', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:41.268244 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-ad2cf33c-b292-44fd-8ea1-28ac457d0a18 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:41.273105 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-d82c0b02-48a4-4a60-aa61-d87a63c95955 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] 10.222.0.16 "GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1" status: 200 len: 1314 microversion: 2.1 time: 0.226006 Oct 30 00:56:41.273694 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 68/142] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:41 2022] GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1 => generated 1314 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:41.277539 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-ad2cf33c-b292-44fd-8ea1-28ac457d0a18 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:41.279066 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-ad2cf33c-b292-44fd-8ea1-28ac457d0a18 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:41.280310 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-ad2cf33c-b292-44fd-8ea1-28ac457d0a18 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1406 microversion: 2.20 time: 0.228552 Oct 30 00:56:41.280940 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 75/143] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 00:56:41 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1406 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 00:56:41.711653 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-092690c5-6bd3-43d6-b5e7-cdf8fc728f40 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:41.712718 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-092690c5-6bd3-43d6-b5e7-cdf8fc728f40 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5652be64-c2a3-4b24-a3aa-c962f204c06e] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:41.719072 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-7b02f712-04e0-4ca2-8352-ae2d15c59019 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:41.719969 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-7b02f712-04e0-4ca2-8352-ae2d15c59019 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:41.721493 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-092690c5-6bd3-43d6-b5e7-cdf8fc728f40 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:41.722182 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-092690c5-6bd3-43d6-b5e7-cdf8fc728f40 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:41.722790 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-092690c5-6bd3-43d6-b5e7-cdf8fc728f40 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:41.723836 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-092690c5-6bd3-43d6-b5e7-cdf8fc728f40 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:41.724545 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-092690c5-6bd3-43d6-b5e7-cdf8fc728f40 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:41.725144 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-092690c5-6bd3-43d6-b5e7-cdf8fc728f40 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:41.727911 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7b02f712-04e0-4ca2-8352-ae2d15c59019 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:41.728390 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7b02f712-04e0-4ca2-8352-ae2d15c59019 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:41.728818 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7b02f712-04e0-4ca2-8352-ae2d15c59019 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:41.729504 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7b02f712-04e0-4ca2-8352-ae2d15c59019 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:41.729944 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7b02f712-04e0-4ca2-8352-ae2d15c59019 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:41.730365 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7b02f712-04e0-4ca2-8352-ae2d15c59019 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:41.924855 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-092690c5-6bd3-43d6-b5e7-cdf8fc728f40 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:41.929969 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-7b02f712-04e0-4ca2-8352-ae2d15c59019 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:41.934109 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-092690c5-6bd3-43d6-b5e7-cdf8fc728f40 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e" status: 200 len: 1367 microversion: 2.1 time: 0.228997 Oct 30 00:56:41.935025 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 69/144] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:41 2022] GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e => generated 1367 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:41.938668 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-7b02f712-04e0-4ca2-8352-ae2d15c59019 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1364 microversion: 2.1 time: 0.224913 Oct 30 00:56:41.939584 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 76/145] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:41 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1364 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:42.296019 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-458ae931-0dfb-420d-8721-dc7c76a993eb tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:42.296639 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-458ae931-0dfb-420d-8721-dc7c76a993eb tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:42.304877 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-9a909139-63cb-4077-b1be-794424d453db tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:42.305478 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-9a909139-63cb-4077-b1be-794424d453db tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:42.306171 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-458ae931-0dfb-420d-8721-dc7c76a993eb tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:42.306654 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-458ae931-0dfb-420d-8721-dc7c76a993eb tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:42.307073 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-458ae931-0dfb-420d-8721-dc7c76a993eb tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:42.307794 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-458ae931-0dfb-420d-8721-dc7c76a993eb tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:42.308230 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-458ae931-0dfb-420d-8721-dc7c76a993eb tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:42.308644 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-458ae931-0dfb-420d-8721-dc7c76a993eb tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:42.311346 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9a909139-63cb-4077-b1be-794424d453db tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:42.311912 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9a909139-63cb-4077-b1be-794424d453db tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:42.312341 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9a909139-63cb-4077-b1be-794424d453db tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:42.313014 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9a909139-63cb-4077-b1be-794424d453db tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:42.313457 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9a909139-63cb-4077-b1be-794424d453db tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:42.313870 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9a909139-63cb-4077-b1be-794424d453db tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:42.506096 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-9a909139-63cb-4077-b1be-794424d453db tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:42.514799 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-458ae931-0dfb-420d-8721-dc7c76a993eb tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c92f7b4d09d74b8d93d739c1938694e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f7d505c7b9e4ca08d6437e6e529a039', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:42.514955 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-9a909139-63cb-4077-b1be-794424d453db tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:42.516341 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-9a909139-63cb-4077-b1be-794424d453db tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:42.517634 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-9a909139-63cb-4077-b1be-794424d453db tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1406 microversion: 2.20 time: 0.216622 Oct 30 00:56:42.519902 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 77/146] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 00:56:42 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1406 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 00:56:42.523004 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-458ae931-0dfb-420d-8721-dc7c76a993eb tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] 10.222.0.16 "GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1" status: 200 len: 1314 microversion: 2.1 time: 0.230846 Oct 30 00:56:42.523764 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 70/147] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:42 2022] GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1 => generated 1314 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:42.963049 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-d8f45528-fa6c-4c31-be61-5b4aebe2edee tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:42.963583 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-ead180f3-2ad2-41e1-a3af-e5a9f4b15388 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:42.963989 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-d8f45528-fa6c-4c31-be61-5b4aebe2edee tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5652be64-c2a3-4b24-a3aa-c962f204c06e] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:42.964203 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-ead180f3-2ad2-41e1-a3af-e5a9f4b15388 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:42.970283 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d8f45528-fa6c-4c31-be61-5b4aebe2edee tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:42.970695 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ead180f3-2ad2-41e1-a3af-e5a9f4b15388 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:42.970845 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d8f45528-fa6c-4c31-be61-5b4aebe2edee tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:42.971186 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d8f45528-fa6c-4c31-be61-5b4aebe2edee tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:42.971330 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ead180f3-2ad2-41e1-a3af-e5a9f4b15388 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:42.971689 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ead180f3-2ad2-41e1-a3af-e5a9f4b15388 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:42.971872 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d8f45528-fa6c-4c31-be61-5b4aebe2edee tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:42.972346 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d8f45528-fa6c-4c31-be61-5b4aebe2edee tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:42.972484 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ead180f3-2ad2-41e1-a3af-e5a9f4b15388 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:42.972746 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d8f45528-fa6c-4c31-be61-5b4aebe2edee tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:42.972839 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ead180f3-2ad2-41e1-a3af-e5a9f4b15388 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:42.973261 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ead180f3-2ad2-41e1-a3af-e5a9f4b15388 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:43.165464 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-d8f45528-fa6c-4c31-be61-5b4aebe2edee tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:43.168652 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-ead180f3-2ad2-41e1-a3af-e5a9f4b15388 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:43.173648 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-d8f45528-fa6c-4c31-be61-5b4aebe2edee tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e" status: 200 len: 1367 microversion: 2.1 time: 0.215484 Oct 30 00:56:43.178490 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 78/148] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:42 2022] GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e => generated 1367 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:43.178624 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-ead180f3-2ad2-41e1-a3af-e5a9f4b15388 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1364 microversion: 2.1 time: 0.218577 Oct 30 00:56:43.178624 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 71/149] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:42 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1364 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:43.543601 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-37f78e5e-fd91-47a5-9606-5f691bc2fc26 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:43.545080 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-37f78e5e-fd91-47a5-9606-5f691bc2fc26 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:43.549629 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-9652994d-bb95-436b-adec-abc4472f5d70 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:43.550264 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-9652994d-bb95-436b-adec-abc4472f5d70 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:43.551910 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-37f78e5e-fd91-47a5-9606-5f691bc2fc26 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:43.552593 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-37f78e5e-fd91-47a5-9606-5f691bc2fc26 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:43.553017 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-37f78e5e-fd91-47a5-9606-5f691bc2fc26 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:43.554469 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-37f78e5e-fd91-47a5-9606-5f691bc2fc26 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:43.555085 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-37f78e5e-fd91-47a5-9606-5f691bc2fc26 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:43.555506 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-37f78e5e-fd91-47a5-9606-5f691bc2fc26 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:43.573445 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9652994d-bb95-436b-adec-abc4472f5d70 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:43.574007 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9652994d-bb95-436b-adec-abc4472f5d70 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:43.574365 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9652994d-bb95-436b-adec-abc4472f5d70 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:43.575096 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9652994d-bb95-436b-adec-abc4472f5d70 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:43.575548 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9652994d-bb95-436b-adec-abc4472f5d70 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:43.575996 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9652994d-bb95-436b-adec-abc4472f5d70 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:43.580866 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell1:SELECT=100 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:56:43.811277 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-9652994d-bb95-436b-adec-abc4472f5d70 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c92f7b4d09d74b8d93d739c1938694e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f7d505c7b9e4ca08d6437e6e529a039', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:43.821355 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-37f78e5e-fd91-47a5-9606-5f691bc2fc26 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:43.831870 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-37f78e5e-fd91-47a5-9606-5f691bc2fc26 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:43.833535 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-37f78e5e-fd91-47a5-9606-5f691bc2fc26 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:43.834178 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-9652994d-bb95-436b-adec-abc4472f5d70 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] 10.222.0.16 "GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1" status: 200 len: 1314 microversion: 2.1 time: 0.288042 Oct 30 00:56:43.835282 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-37f78e5e-fd91-47a5-9606-5f691bc2fc26 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1406 microversion: 2.20 time: 0.295705 Oct 30 00:56:43.836307 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 72/150] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:43 2022] GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1 => generated 1314 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:43.836422 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 79/151] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 00:56:43 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1406 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 00:56:44.084176 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [req-e4892e6a-0219-4b54-abea-a12e3aa1e41d req-df069b36-2d9a-4fa9-8d1f-1cdb2a5071aa service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0f858c8d-6414-403e-b8d3-3ecd728e84a1", "name": "network-vif-plugged", "status": "completed", "tag": "2c0a4a56-006d-4324-b26e-f48bac7f7a5b"}]} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:56:44.094427 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-e4892e6a-0219-4b54-abea-a12e3aa1e41d req-df069b36-2d9a-4fa9-8d1f-1cdb2a5071aa service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:44.094913 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-e4892e6a-0219-4b54-abea-a12e3aa1e41d req-df069b36-2d9a-4fa9-8d1f-1cdb2a5071aa service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:44.095525 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-e4892e6a-0219-4b54-abea-a12e3aa1e41d req-df069b36-2d9a-4fa9-8d1f-1cdb2a5071aa service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:44.108424 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.compute.server_external_events [req-e4892e6a-0219-4b54-abea-a12e3aa1e41d req-df069b36-2d9a-4fa9-8d1f-1cdb2a5071aa service nova] Creating event network-vif-plugged:2c0a4a56-006d-4324-b26e-f48bac7f7a5b for instance 0f858c8d-6414-403e-b8d3-3ecd728e84a1 on hv-cinder-maste Oct 30 00:56:44.113770 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [req-e4892e6a-0219-4b54-abea-a12e3aa1e41d req-df069b36-2d9a-4fa9-8d1f-1cdb2a5071aa service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.091362 Oct 30 00:56:44.114350 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 73/152] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 00:56:44 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:56:44.196825 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-5e604bf6-68d4-451f-9a64-fc899a41917c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:44.197780 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-5e604bf6-68d4-451f-9a64-fc899a41917c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5652be64-c2a3-4b24-a3aa-c962f204c06e] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:44.203944 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-2e8d9720-5cd4-4d61-b363-de92f1445834 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:44.204412 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5e604bf6-68d4-451f-9a64-fc899a41917c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:44.204961 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5e604bf6-68d4-451f-9a64-fc899a41917c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:44.205306 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-2e8d9720-5cd4-4d61-b363-de92f1445834 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:44.205566 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5e604bf6-68d4-451f-9a64-fc899a41917c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:44.206315 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5e604bf6-68d4-451f-9a64-fc899a41917c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:44.206783 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5e604bf6-68d4-451f-9a64-fc899a41917c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:44.207203 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5e604bf6-68d4-451f-9a64-fc899a41917c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:44.212192 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2e8d9720-5cd4-4d61-b363-de92f1445834 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:44.212739 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2e8d9720-5cd4-4d61-b363-de92f1445834 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:44.213367 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2e8d9720-5cd4-4d61-b363-de92f1445834 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:44.214268 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2e8d9720-5cd4-4d61-b363-de92f1445834 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:44.214751 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2e8d9720-5cd4-4d61-b363-de92f1445834 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:44.215391 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2e8d9720-5cd4-4d61-b363-de92f1445834 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:44.414827 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-2e8d9720-5cd4-4d61-b363-de92f1445834 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:44.420256 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-5e604bf6-68d4-451f-9a64-fc899a41917c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:44.423253 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-2e8d9720-5cd4-4d61-b363-de92f1445834 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1364 microversion: 2.1 time: 0.224220 Oct 30 00:56:44.424066 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 74/153] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:44 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1364 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 00:56:44.430379 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-5e604bf6-68d4-451f-9a64-fc899a41917c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e" status: 200 len: 1367 microversion: 2.1 time: 0.237841 Oct 30 00:56:44.430964 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 80/154] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:44 2022] GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e => generated 1367 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:44.858315 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-f779f514-c32b-4bcf-91d4-393c1e50d812 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:44.858925 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-f779f514-c32b-4bcf-91d4-393c1e50d812 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:44.865088 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f779f514-c32b-4bcf-91d4-393c1e50d812 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:44.865278 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-4fd7afea-bd7a-42e0-a244-7f8de3eeaf77 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:44.865585 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f779f514-c32b-4bcf-91d4-393c1e50d812 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:44.865860 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-4fd7afea-bd7a-42e0-a244-7f8de3eeaf77 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:44.866044 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f779f514-c32b-4bcf-91d4-393c1e50d812 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:44.866897 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f779f514-c32b-4bcf-91d4-393c1e50d812 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:44.867406 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f779f514-c32b-4bcf-91d4-393c1e50d812 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:44.867775 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f779f514-c32b-4bcf-91d4-393c1e50d812 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:44.876042 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4fd7afea-bd7a-42e0-a244-7f8de3eeaf77 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:44.876638 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4fd7afea-bd7a-42e0-a244-7f8de3eeaf77 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:44.877152 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4fd7afea-bd7a-42e0-a244-7f8de3eeaf77 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:44.877917 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4fd7afea-bd7a-42e0-a244-7f8de3eeaf77 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:44.878553 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4fd7afea-bd7a-42e0-a244-7f8de3eeaf77 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:44.879044 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4fd7afea-bd7a-42e0-a244-7f8de3eeaf77 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:45.232725 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-f779f514-c32b-4bcf-91d4-393c1e50d812 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c92f7b4d09d74b8d93d739c1938694e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f7d505c7b9e4ca08d6437e6e529a039', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:45.237202 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-4fd7afea-bd7a-42e0-a244-7f8de3eeaf77 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:45.242716 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-f779f514-c32b-4bcf-91d4-393c1e50d812 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] 10.222.0.16 "GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1" status: 200 len: 1314 microversion: 2.1 time: 0.388188 Oct 30 00:56:45.243709 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 75/155] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:44 2022] GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1 => generated 1314 bytes in 390 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:45.252614 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-4fd7afea-bd7a-42e0-a244-7f8de3eeaf77 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:45.254193 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-4fd7afea-bd7a-42e0-a244-7f8de3eeaf77 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:45.255372 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-4fd7afea-bd7a-42e0-a244-7f8de3eeaf77 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1406 microversion: 2.20 time: 0.394246 Oct 30 00:56:45.256028 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 81/156] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 00:56:44 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1406 bytes in 395 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 00:56:45.447300 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-21ebc533-eeda-49c0-949a-e7d4eb80714c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:45.448714 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-21ebc533-eeda-49c0-949a-e7d4eb80714c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:45.452757 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-f668b1de-412e-4eb7-88d5-3bb2505d7244 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:45.453358 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-f668b1de-412e-4eb7-88d5-3bb2505d7244 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5652be64-c2a3-4b24-a3aa-c962f204c06e] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:45.455218 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-21ebc533-eeda-49c0-949a-e7d4eb80714c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:45.455704 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-21ebc533-eeda-49c0-949a-e7d4eb80714c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:45.456166 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-21ebc533-eeda-49c0-949a-e7d4eb80714c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:45.456854 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-21ebc533-eeda-49c0-949a-e7d4eb80714c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:45.457296 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-21ebc533-eeda-49c0-949a-e7d4eb80714c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:45.457711 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-21ebc533-eeda-49c0-949a-e7d4eb80714c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:45.459324 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f668b1de-412e-4eb7-88d5-3bb2505d7244 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:45.459833 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f668b1de-412e-4eb7-88d5-3bb2505d7244 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:45.460354 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f668b1de-412e-4eb7-88d5-3bb2505d7244 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:45.460926 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f668b1de-412e-4eb7-88d5-3bb2505d7244 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:45.461414 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f668b1de-412e-4eb7-88d5-3bb2505d7244 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:45.463070 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f668b1de-412e-4eb7-88d5-3bb2505d7244 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:45.673972 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-21ebc533-eeda-49c0-949a-e7d4eb80714c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:45.679496 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-f668b1de-412e-4eb7-88d5-3bb2505d7244 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:45.682794 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-21ebc533-eeda-49c0-949a-e7d4eb80714c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1364 microversion: 2.1 time: 0.240023 Oct 30 00:56:45.683372 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 76/157] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:45 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1364 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:45.700072 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-f668b1de-412e-4eb7-88d5-3bb2505d7244 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e" status: 200 len: 1367 microversion: 2.1 time: 0.250939 Oct 30 00:56:45.700792 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 82/158] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:45 2022] GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e => generated 1367 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 00:56:46.269593 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-2508be0a-165a-49bb-b154-ae78ee55a62d tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:46.270205 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-2508be0a-165a-49bb-b154-ae78ee55a62d tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:46.283278 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-581ac808-87f7-465e-a89e-97f9ebdc0012 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:46.284554 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-581ac808-87f7-465e-a89e-97f9ebdc0012 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:46.287794 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2508be0a-165a-49bb-b154-ae78ee55a62d tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:46.288286 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2508be0a-165a-49bb-b154-ae78ee55a62d tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:46.288722 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2508be0a-165a-49bb-b154-ae78ee55a62d tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:46.289449 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2508be0a-165a-49bb-b154-ae78ee55a62d tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:46.289900 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2508be0a-165a-49bb-b154-ae78ee55a62d tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:46.290575 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2508be0a-165a-49bb-b154-ae78ee55a62d tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:46.294621 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-581ac808-87f7-465e-a89e-97f9ebdc0012 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:46.295513 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-581ac808-87f7-465e-a89e-97f9ebdc0012 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:46.296140 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-581ac808-87f7-465e-a89e-97f9ebdc0012 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:46.297018 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-581ac808-87f7-465e-a89e-97f9ebdc0012 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:46.297472 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-581ac808-87f7-465e-a89e-97f9ebdc0012 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:46.297887 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-581ac808-87f7-465e-a89e-97f9ebdc0012 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:46.646211 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-581ac808-87f7-465e-a89e-97f9ebdc0012 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:46.657429 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-581ac808-87f7-465e-a89e-97f9ebdc0012 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:46.661215 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-581ac808-87f7-465e-a89e-97f9ebdc0012 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:46.664764 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-581ac808-87f7-465e-a89e-97f9ebdc0012 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1406 microversion: 2.20 time: 0.387726 Oct 30 00:56:46.667653 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 83/159] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 00:56:46 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1406 bytes in 391 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 00:56:46.692283 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-2508be0a-165a-49bb-b154-ae78ee55a62d tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c92f7b4d09d74b8d93d739c1938694e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f7d505c7b9e4ca08d6437e6e529a039', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:46.708935 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-78904f80-44b6-4b1c-a9dc-71daec676842 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:46.710416 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-78904f80-44b6-4b1c-a9dc-71daec676842 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:46.725463 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-2508be0a-165a-49bb-b154-ae78ee55a62d tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] 10.222.0.16 "GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1" status: 200 len: 1314 microversion: 2.1 time: 0.458279 Oct 30 00:56:46.726657 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-78904f80-44b6-4b1c-a9dc-71daec676842 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:46.741948 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 77/160] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:46 2022] GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1 => generated 1314 bytes in 476 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:46.748084 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-78904f80-44b6-4b1c-a9dc-71daec676842 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:46.748645 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-78904f80-44b6-4b1c-a9dc-71daec676842 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.021s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:46.749402 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-78904f80-44b6-4b1c-a9dc-71daec676842 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:46.749867 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-78904f80-44b6-4b1c-a9dc-71daec676842 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:46.751172 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-78904f80-44b6-4b1c-a9dc-71daec676842 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:46.757279 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-9923dbdf-93ec-4f35-824c-4ed537624adc tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:46.758164 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-9923dbdf-93ec-4f35-824c-4ed537624adc tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5652be64-c2a3-4b24-a3aa-c962f204c06e] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:46.773462 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9923dbdf-93ec-4f35-824c-4ed537624adc tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:46.777752 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9923dbdf-93ec-4f35-824c-4ed537624adc tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:46.780339 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9923dbdf-93ec-4f35-824c-4ed537624adc tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:46.781242 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9923dbdf-93ec-4f35-824c-4ed537624adc tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:46.783048 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9923dbdf-93ec-4f35-824c-4ed537624adc tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:46.783928 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9923dbdf-93ec-4f35-824c-4ed537624adc tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:46.787633 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell1:SELECT=100 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:56:46.971287 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-78904f80-44b6-4b1c-a9dc-71daec676842 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:46.985531 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-78904f80-44b6-4b1c-a9dc-71daec676842 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1364 microversion: 2.1 time: 0.281576 Oct 30 00:56:46.986137 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 84/161] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:46 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1364 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:47.081235 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-9923dbdf-93ec-4f35-824c-4ed537624adc tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:47.094256 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-9923dbdf-93ec-4f35-824c-4ed537624adc tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e" status: 200 len: 1367 microversion: 2.1 time: 0.350610 Oct 30 00:56:47.096693 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 78/162] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:46 2022] GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e => generated 1367 bytes in 354 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:47.697191 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-8fe00428-9bdd-43ed-ae5d-c6d0a1e25bd6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:47.697968 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-8fe00428-9bdd-43ed-ae5d-c6d0a1e25bd6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:47.714395 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8fe00428-9bdd-43ed-ae5d-c6d0a1e25bd6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:47.714900 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8fe00428-9bdd-43ed-ae5d-c6d0a1e25bd6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:47.715328 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8fe00428-9bdd-43ed-ae5d-c6d0a1e25bd6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:47.716064 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8fe00428-9bdd-43ed-ae5d-c6d0a1e25bd6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:47.716655 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8fe00428-9bdd-43ed-ae5d-c6d0a1e25bd6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:47.717240 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8fe00428-9bdd-43ed-ae5d-c6d0a1e25bd6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:47.755078 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-73581f6c-6f2e-4eb8-9d54-8e8990a94b39 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:47.755685 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-73581f6c-6f2e-4eb8-9d54-8e8990a94b39 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:47.766044 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-73581f6c-6f2e-4eb8-9d54-8e8990a94b39 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:47.766803 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-73581f6c-6f2e-4eb8-9d54-8e8990a94b39 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:47.767400 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-73581f6c-6f2e-4eb8-9d54-8e8990a94b39 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:47.768872 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-73581f6c-6f2e-4eb8-9d54-8e8990a94b39 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:47.769613 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-73581f6c-6f2e-4eb8-9d54-8e8990a94b39 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:47.770194 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-73581f6c-6f2e-4eb8-9d54-8e8990a94b39 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:48.015924 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-8fe00428-9bdd-43ed-ae5d-c6d0a1e25bd6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:48.027218 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-8fe00428-9bdd-43ed-ae5d-c6d0a1e25bd6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:48.029849 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-8fe00428-9bdd-43ed-ae5d-c6d0a1e25bd6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:48.032381 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-8fe00428-9bdd-43ed-ae5d-c6d0a1e25bd6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1406 microversion: 2.20 time: 0.338802 Oct 30 00:56:48.034653 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 85/163] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 00:56:47 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1406 bytes in 343 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 00:56:48.039576 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-73581f6c-6f2e-4eb8-9d54-8e8990a94b39 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c92f7b4d09d74b8d93d739c1938694e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f7d505c7b9e4ca08d6437e6e529a039', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:48.041604 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-52346073-137b-4189-bb34-fb19607bc0f5 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:48.042721 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-52346073-137b-4189-bb34-fb19607bc0f5 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:48.057815 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-52346073-137b-4189-bb34-fb19607bc0f5 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:48.058533 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-52346073-137b-4189-bb34-fb19607bc0f5 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:48.059096 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-73581f6c-6f2e-4eb8-9d54-8e8990a94b39 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] 10.222.0.16 "GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1" status: 200 len: 1314 microversion: 2.1 time: 0.308160 Oct 30 00:56:48.059370 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-52346073-137b-4189-bb34-fb19607bc0f5 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:48.060457 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-52346073-137b-4189-bb34-fb19607bc0f5 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:48.061119 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-52346073-137b-4189-bb34-fb19607bc0f5 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:48.061639 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 79/164] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:47 2022] GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1 => generated 1314 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:48.061880 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-52346073-137b-4189-bb34-fb19607bc0f5 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:48.162283 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-33a30fdb-361b-450b-9549-4a2ad92830b2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:48.165187 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-33a30fdb-361b-450b-9549-4a2ad92830b2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5652be64-c2a3-4b24-a3aa-c962f204c06e] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:48.177564 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-33a30fdb-361b-450b-9549-4a2ad92830b2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:48.179441 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-33a30fdb-361b-450b-9549-4a2ad92830b2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:48.180484 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-33a30fdb-361b-450b-9549-4a2ad92830b2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:48.182484 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-33a30fdb-361b-450b-9549-4a2ad92830b2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:48.183580 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-33a30fdb-361b-450b-9549-4a2ad92830b2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:48.184649 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-33a30fdb-361b-450b-9549-4a2ad92830b2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:48.484053 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-52346073-137b-4189-bb34-fb19607bc0f5 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:48.486035 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-33a30fdb-361b-450b-9549-4a2ad92830b2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:48.495898 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-52346073-137b-4189-bb34-fb19607bc0f5 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1364 microversion: 2.1 time: 0.459593 Oct 30 00:56:48.496053 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-33a30fdb-361b-450b-9549-4a2ad92830b2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e" status: 200 len: 1367 microversion: 2.1 time: 0.377608 Oct 30 00:56:48.496648 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 86/165] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:48 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1364 bytes in 461 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:48.496855 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 80/166] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:48 2022] GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e => generated 1367 bytes in 379 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:48.511354 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [req-daabcde5-127b-4db5-ba73-b8c26b69b1b7 req-d121fbe4-f3c3-4fb8-9e24-5e6a99019af6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5652be64-c2a3-4b24-a3aa-c962f204c06e", "name": "network-vif-plugged", "status": "completed", "tag": "9218be5d-9ff8-4e15-b073-e87bc5e0401c"}]} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:56:48.539880 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-daabcde5-127b-4db5-ba73-b8c26b69b1b7 req-d121fbe4-f3c3-4fb8-9e24-5e6a99019af6 service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:48.540351 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-daabcde5-127b-4db5-ba73-b8c26b69b1b7 req-d121fbe4-f3c3-4fb8-9e24-5e6a99019af6 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:48.540775 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-daabcde5-127b-4db5-ba73-b8c26b69b1b7 req-d121fbe4-f3c3-4fb8-9e24-5e6a99019af6 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:48.571042 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.compute.server_external_events [req-daabcde5-127b-4db5-ba73-b8c26b69b1b7 req-d121fbe4-f3c3-4fb8-9e24-5e6a99019af6 service nova] Creating event network-vif-plugged:9218be5d-9ff8-4e15-b073-e87bc5e0401c for instance 5652be64-c2a3-4b24-a3aa-c962f204c06e on hv-cinder-maste Oct 30 00:56:48.571786 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-daabcde5-127b-4db5-ba73-b8c26b69b1b7 req-d121fbe4-f3c3-4fb8-9e24-5e6a99019af6 service nova] Acquiring lock "compute-rpcapi-router" {{(pid=118962) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Oct 30 00:56:48.572498 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-daabcde5-127b-4db5-ba73-b8c26b69b1b7 req-d121fbe4-f3c3-4fb8-9e24-5e6a99019af6 service nova] Acquired lock "compute-rpcapi-router" {{(pid=118962) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Oct 30 00:56:48.573897 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-58f726bb-b247-44fe-8ba5-2fc812f5833c None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:48.574935 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-58f726bb-b247-44fe-8ba5-2fc812f5833c None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:48.575978 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-58f726bb-b247-44fe-8ba5-2fc812f5833c None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:48.577613 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-58f726bb-b247-44fe-8ba5-2fc812f5833c None None] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:48.578318 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-58f726bb-b247-44fe-8ba5-2fc812f5833c None None] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:48.579336 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-58f726bb-b247-44fe-8ba5-2fc812f5833c None None] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:48.592494 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.compute.rpcapi [None req-58f726bb-b247-44fe-8ba5-2fc812f5833c None None] Automatically selected compute RPC version 6.1 from minimum service version 64 Oct 30 00:56:48.593646 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-58f726bb-b247-44fe-8ba5-2fc812f5833c None None] Releasing lock "compute-rpcapi-router" {{(pid=118962) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Oct 30 00:56:48.622579 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-58f726bb-b247-44fe-8ba5-2fc812f5833c None None] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.124338 Oct 30 00:56:48.623656 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 87/167] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 00:56:48 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:56:49.063088 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-bb0af613-f5d4-43d0-92d7-b44a1abf4b89 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:49.064303 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-bb0af613-f5d4-43d0-92d7-b44a1abf4b89 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:49.073014 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_api:SELECT=97 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:56:49.083831 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-ce6ac083-48fd-4541-91f3-c20ca0d69a36 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:49.084700 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-ce6ac083-48fd-4541-91f3-c20ca0d69a36 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:49.086878 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bb0af613-f5d4-43d0-92d7-b44a1abf4b89 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:49.087642 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bb0af613-f5d4-43d0-92d7-b44a1abf4b89 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:49.089498 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bb0af613-f5d4-43d0-92d7-b44a1abf4b89 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:49.091515 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bb0af613-f5d4-43d0-92d7-b44a1abf4b89 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:49.092430 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bb0af613-f5d4-43d0-92d7-b44a1abf4b89 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:49.093268 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ce6ac083-48fd-4541-91f3-c20ca0d69a36 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:49.093677 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bb0af613-f5d4-43d0-92d7-b44a1abf4b89 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:49.094217 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ce6ac083-48fd-4541-91f3-c20ca0d69a36 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:49.094846 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ce6ac083-48fd-4541-91f3-c20ca0d69a36 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:49.096346 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ce6ac083-48fd-4541-91f3-c20ca0d69a36 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:49.096943 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ce6ac083-48fd-4541-91f3-c20ca0d69a36 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:49.097542 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ce6ac083-48fd-4541-91f3-c20ca0d69a36 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:49.350744 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-bb0af613-f5d4-43d0-92d7-b44a1abf4b89 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:49.355426 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-ce6ac083-48fd-4541-91f3-c20ca0d69a36 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c92f7b4d09d74b8d93d739c1938694e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f7d505c7b9e4ca08d6437e6e529a039', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:49.360071 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-bb0af613-f5d4-43d0-92d7-b44a1abf4b89 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:49.361522 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-bb0af613-f5d4-43d0-92d7-b44a1abf4b89 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:49.366669 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-ce6ac083-48fd-4541-91f3-c20ca0d69a36 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] 10.222.0.16 "GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1" status: 200 len: 1314 microversion: 2.1 time: 0.287090 Oct 30 00:56:49.367415 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 88/168] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:49 2022] GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1 => generated 1314 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 00:56:49.371050 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-bb0af613-f5d4-43d0-92d7-b44a1abf4b89 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1406 microversion: 2.20 time: 0.311840 Oct 30 00:56:49.371957 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 81/169] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 00:56:49 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1406 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 00:56:49.521018 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-cf5172bf-de4d-4462-a988-5b6d9384cc41 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:49.521431 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-214ae67f-9f81-433e-b3b0-b5f799e60dda tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:49.522044 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-cf5172bf-de4d-4462-a988-5b6d9384cc41 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5652be64-c2a3-4b24-a3aa-c962f204c06e] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:49.522631 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-214ae67f-9f81-433e-b3b0-b5f799e60dda tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:49.528990 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-214ae67f-9f81-433e-b3b0-b5f799e60dda tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:49.529478 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-214ae67f-9f81-433e-b3b0-b5f799e60dda tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:49.529901 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-214ae67f-9f81-433e-b3b0-b5f799e60dda tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:49.530596 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-214ae67f-9f81-433e-b3b0-b5f799e60dda tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:49.531034 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-214ae67f-9f81-433e-b3b0-b5f799e60dda tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:49.531766 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-214ae67f-9f81-433e-b3b0-b5f799e60dda tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:49.533458 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-cf5172bf-de4d-4462-a988-5b6d9384cc41 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:49.535158 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-cf5172bf-de4d-4462-a988-5b6d9384cc41 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:49.535664 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-cf5172bf-de4d-4462-a988-5b6d9384cc41 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:49.536456 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-cf5172bf-de4d-4462-a988-5b6d9384cc41 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:49.539482 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-cf5172bf-de4d-4462-a988-5b6d9384cc41 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:49.540005 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-cf5172bf-de4d-4462-a988-5b6d9384cc41 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:49.782761 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-cf5172bf-de4d-4462-a988-5b6d9384cc41 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:49.787749 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-214ae67f-9f81-433e-b3b0-b5f799e60dda tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:49.793232 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-cf5172bf-de4d-4462-a988-5b6d9384cc41 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e" status: 200 len: 1367 microversion: 2.1 time: 0.276851 Oct 30 00:56:49.793958 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 89/170] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:49 2022] GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e => generated 1367 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:49.796591 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-214ae67f-9f81-433e-b3b0-b5f799e60dda tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1364 microversion: 2.1 time: 0.280007 Oct 30 00:56:49.800634 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 82/171] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:49 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1364 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:50.393399 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-056811bd-5695-4bd2-9d6b-541e19af35fd tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:50.394907 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-056811bd-5695-4bd2-9d6b-541e19af35fd tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:50.402486 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-056811bd-5695-4bd2-9d6b-541e19af35fd tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:50.403319 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-056811bd-5695-4bd2-9d6b-541e19af35fd tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:50.403986 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-056811bd-5695-4bd2-9d6b-541e19af35fd tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:50.404880 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-056811bd-5695-4bd2-9d6b-541e19af35fd tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:50.405518 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-056811bd-5695-4bd2-9d6b-541e19af35fd tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:50.406405 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-056811bd-5695-4bd2-9d6b-541e19af35fd tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:50.416614 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-07a6be35-3cf1-464f-9f3c-8b3d6cfc82f6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:50.417290 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-07a6be35-3cf1-464f-9f3c-8b3d6cfc82f6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:50.424789 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-07a6be35-3cf1-464f-9f3c-8b3d6cfc82f6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:50.425559 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-07a6be35-3cf1-464f-9f3c-8b3d6cfc82f6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:50.426464 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-07a6be35-3cf1-464f-9f3c-8b3d6cfc82f6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:50.427387 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-07a6be35-3cf1-464f-9f3c-8b3d6cfc82f6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:50.428164 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-07a6be35-3cf1-464f-9f3c-8b3d6cfc82f6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:50.428967 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-07a6be35-3cf1-464f-9f3c-8b3d6cfc82f6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:50.762592 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-07a6be35-3cf1-464f-9f3c-8b3d6cfc82f6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:50.775196 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-056811bd-5695-4bd2-9d6b-541e19af35fd tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c92f7b4d09d74b8d93d739c1938694e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f7d505c7b9e4ca08d6437e6e529a039', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:50.781054 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-07a6be35-3cf1-464f-9f3c-8b3d6cfc82f6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:50.782488 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-07a6be35-3cf1-464f-9f3c-8b3d6cfc82f6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:50.783613 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-07a6be35-3cf1-464f-9f3c-8b3d6cfc82f6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1406 microversion: 2.20 time: 0.390433 Oct 30 00:56:50.784489 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 83/172] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 00:56:50 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1406 bytes in 392 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 00:56:50.793846 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-056811bd-5695-4bd2-9d6b-541e19af35fd tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] 10.222.0.16 "GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1" status: 200 len: 1314 microversion: 2.1 time: 0.407017 Oct 30 00:56:50.795442 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 90/173] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:50 2022] GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1 => generated 1314 bytes in 409 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:50.811582 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [req-79496e6f-fcad-437c-8e53-f285d0d64195 req-36e289f2-c23c-4f27-aa75-99e4113f96d5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9491a8f4-2b76-4a4e-a6bd-6720114e482f", "name": "network-vif-plugged", "status": "completed", "tag": "5367850e-73d0-4814-af96-bd430c94c272"}]} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:56:50.822658 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-05523d58-3a5b-47b2-ad5a-00338e877743 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:50.823507 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-05523d58-3a5b-47b2-ad5a-00338e877743 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:50.828160 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-79496e6f-fcad-437c-8e53-f285d0d64195 req-36e289f2-c23c-4f27-aa75-99e4113f96d5 service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:50.828588 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-79496e6f-fcad-437c-8e53-f285d0d64195 req-36e289f2-c23c-4f27-aa75-99e4113f96d5 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:50.828993 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-79496e6f-fcad-437c-8e53-f285d0d64195 req-36e289f2-c23c-4f27-aa75-99e4113f96d5 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:50.841978 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-05523d58-3a5b-47b2-ad5a-00338e877743 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:50.849030 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-05523d58-3a5b-47b2-ad5a-00338e877743 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:50.849688 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-05523d58-3a5b-47b2-ad5a-00338e877743 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:50.850788 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-05523d58-3a5b-47b2-ad5a-00338e877743 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:50.851422 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-05523d58-3a5b-47b2-ad5a-00338e877743 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:50.852066 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-05523d58-3a5b-47b2-ad5a-00338e877743 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:50.854093 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.compute.server_external_events [req-79496e6f-fcad-437c-8e53-f285d0d64195 req-36e289f2-c23c-4f27-aa75-99e4113f96d5 service nova] Creating event network-vif-plugged:5367850e-73d0-4814-af96-bd430c94c272 for instance 9491a8f4-2b76-4a4e-a6bd-6720114e482f on hv-cinder-maste Oct 30 00:56:50.862269 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [req-79496e6f-fcad-437c-8e53-f285d0d64195 req-36e289f2-c23c-4f27-aa75-99e4113f96d5 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.060355 Oct 30 00:56:50.863294 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 84/174] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 00:56:50 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:56:50.869609 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-1b5e679c-c481-4862-bf8e-998600f22885 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:50.870656 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-1b5e679c-c481-4862-bf8e-998600f22885 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5652be64-c2a3-4b24-a3aa-c962f204c06e] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:50.879322 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1b5e679c-c481-4862-bf8e-998600f22885 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:50.880612 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1b5e679c-c481-4862-bf8e-998600f22885 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:50.881352 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1b5e679c-c481-4862-bf8e-998600f22885 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:50.882297 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1b5e679c-c481-4862-bf8e-998600f22885 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:50.882744 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1b5e679c-c481-4862-bf8e-998600f22885 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:50.883156 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1b5e679c-c481-4862-bf8e-998600f22885 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:51.093815 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-1b5e679c-c481-4862-bf8e-998600f22885 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:51.104240 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-1b5e679c-c481-4862-bf8e-998600f22885 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e" status: 200 len: 1367 microversion: 2.1 time: 0.239699 Oct 30 00:56:51.104850 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-05523d58-3a5b-47b2-ad5a-00338e877743 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:51.105174 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 85/175] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:50 2022] GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e => generated 1367 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:51.114529 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-05523d58-3a5b-47b2-ad5a-00338e877743 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1364 microversion: 2.1 time: 0.297885 Oct 30 00:56:51.130694 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 91/176] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:50 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1364 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:51.813942 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-5f3181d3-0858-48f6-bf2b-7cf47ed5a7b2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:51.814729 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-5f3181d3-0858-48f6-bf2b-7cf47ed5a7b2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:51.821290 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-b77dc51a-5458-403a-9366-198b7ec9ffd6 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:51.822171 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-b77dc51a-5458-403a-9366-198b7ec9ffd6 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:51.822776 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5f3181d3-0858-48f6-bf2b-7cf47ed5a7b2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:51.823489 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5f3181d3-0858-48f6-bf2b-7cf47ed5a7b2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:51.824153 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5f3181d3-0858-48f6-bf2b-7cf47ed5a7b2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:51.825050 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5f3181d3-0858-48f6-bf2b-7cf47ed5a7b2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:51.825940 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5f3181d3-0858-48f6-bf2b-7cf47ed5a7b2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:51.826826 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5f3181d3-0858-48f6-bf2b-7cf47ed5a7b2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:51.830772 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b77dc51a-5458-403a-9366-198b7ec9ffd6 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:51.831273 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b77dc51a-5458-403a-9366-198b7ec9ffd6 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:51.831988 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b77dc51a-5458-403a-9366-198b7ec9ffd6 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:51.832731 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b77dc51a-5458-403a-9366-198b7ec9ffd6 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:51.833473 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b77dc51a-5458-403a-9366-198b7ec9ffd6 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:51.834010 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b77dc51a-5458-403a-9366-198b7ec9ffd6 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:52.037794 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-b77dc51a-5458-403a-9366-198b7ec9ffd6 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c92f7b4d09d74b8d93d739c1938694e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f7d505c7b9e4ca08d6437e6e529a039', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:52.050876 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-b77dc51a-5458-403a-9366-198b7ec9ffd6 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] 10.222.0.16 "GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1" status: 200 len: 1314 microversion: 2.1 time: 0.233135 Oct 30 00:56:52.063753 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 92/177] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:51 2022] GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1 => generated 1314 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:52.130101 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-24c96edc-6040-4e01-b2c5-abbc09acbb20 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:52.133077 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-24c96edc-6040-4e01-b2c5-abbc09acbb20 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5652be64-c2a3-4b24-a3aa-c962f204c06e] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:52.148690 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-5f3181d3-0858-48f6-bf2b-7cf47ed5a7b2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:52.155272 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-24c96edc-6040-4e01-b2c5-abbc09acbb20 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:52.157251 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-24c96edc-6040-4e01-b2c5-abbc09acbb20 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:52.157886 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-24c96edc-6040-4e01-b2c5-abbc09acbb20 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:52.158862 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-24c96edc-6040-4e01-b2c5-abbc09acbb20 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:52.159584 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-24c96edc-6040-4e01-b2c5-abbc09acbb20 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:52.160258 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-5f3181d3-0858-48f6-bf2b-7cf47ed5a7b2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:52.160537 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-24c96edc-6040-4e01-b2c5-abbc09acbb20 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:52.162071 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-5f3181d3-0858-48f6-bf2b-7cf47ed5a7b2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:52.163800 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-5f3181d3-0858-48f6-bf2b-7cf47ed5a7b2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1406 microversion: 2.20 time: 0.353907 Oct 30 00:56:52.175506 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 86/178] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 00:56:51 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1406 bytes in 355 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 00:56:52.175506 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-8d448868-36b1-4389-a349-c81f2f2deb84 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:52.175506 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-8d448868-36b1-4389-a349-c81f2f2deb84 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:52.177520 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8d448868-36b1-4389-a349-c81f2f2deb84 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:52.178122 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8d448868-36b1-4389-a349-c81f2f2deb84 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:52.180765 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8d448868-36b1-4389-a349-c81f2f2deb84 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:52.182705 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8d448868-36b1-4389-a349-c81f2f2deb84 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:52.183276 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8d448868-36b1-4389-a349-c81f2f2deb84 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:52.183834 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8d448868-36b1-4389-a349-c81f2f2deb84 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:52.461497 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-8d448868-36b1-4389-a349-c81f2f2deb84 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:52.466299 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-24c96edc-6040-4e01-b2c5-abbc09acbb20 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:52.472241 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-8d448868-36b1-4389-a349-c81f2f2deb84 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1364 microversion: 2.1 time: 0.306206 Oct 30 00:56:52.477938 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-24c96edc-6040-4e01-b2c5-abbc09acbb20 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e" status: 200 len: 1367 microversion: 2.1 time: 0.353124 Oct 30 00:56:52.478997 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 93/180] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:52 2022] GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e => generated 1367 bytes in 355 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:52.480337 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 87/180] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:52 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1364 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:53.078357 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-33a1dbc9-3b00-42e3-b16b-1ac6f319c87e tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:53.079636 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-33a1dbc9-3b00-42e3-b16b-1ac6f319c87e tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:53.086848 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-33a1dbc9-3b00-42e3-b16b-1ac6f319c87e tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:53.087908 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-33a1dbc9-3b00-42e3-b16b-1ac6f319c87e tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:53.088492 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-33a1dbc9-3b00-42e3-b16b-1ac6f319c87e tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:53.089338 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-33a1dbc9-3b00-42e3-b16b-1ac6f319c87e tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:53.089929 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-33a1dbc9-3b00-42e3-b16b-1ac6f319c87e tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:53.090660 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-33a1dbc9-3b00-42e3-b16b-1ac6f319c87e tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:53.193617 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-abf09b7a-998c-45a4-9365-3f23078a52cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:53.195745 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-abf09b7a-998c-45a4-9365-3f23078a52cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:53.202365 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-abf09b7a-998c-45a4-9365-3f23078a52cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:53.203266 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-abf09b7a-998c-45a4-9365-3f23078a52cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:53.203898 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-abf09b7a-998c-45a4-9365-3f23078a52cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:53.204910 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-abf09b7a-998c-45a4-9365-3f23078a52cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:53.205358 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-abf09b7a-998c-45a4-9365-3f23078a52cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:53.206101 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-abf09b7a-998c-45a4-9365-3f23078a52cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:53.316773 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-33a1dbc9-3b00-42e3-b16b-1ac6f319c87e tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c92f7b4d09d74b8d93d739c1938694e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f7d505c7b9e4ca08d6437e6e529a039', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:53.329035 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-33a1dbc9-3b00-42e3-b16b-1ac6f319c87e tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] 10.222.0.16 "GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1" status: 200 len: 1314 microversion: 2.1 time: 0.254351 Oct 30 00:56:53.336683 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 94/181] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:53 2022] GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1 => generated 1314 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:53.497200 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-abf09b7a-998c-45a4-9365-3f23078a52cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:53.499837 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-f2248211-2aad-42b9-a97a-7b06c5da9ed1 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:53.501311 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-f2248211-2aad-42b9-a97a-7b06c5da9ed1 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:53.509143 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-abf09b7a-998c-45a4-9365-3f23078a52cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:53.512594 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-abf09b7a-998c-45a4-9365-3f23078a52cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:53.513779 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f2248211-2aad-42b9-a97a-7b06c5da9ed1 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:53.513892 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-abf09b7a-998c-45a4-9365-3f23078a52cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1406 microversion: 2.20 time: 0.324282 Oct 30 00:56:53.514393 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 88/182] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 00:56:53 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1406 bytes in 326 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 30 00:56:53.514798 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f2248211-2aad-42b9-a97a-7b06c5da9ed1 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:53.516068 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f2248211-2aad-42b9-a97a-7b06c5da9ed1 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:53.519944 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-754d326e-8192-4c63-8ac6-68ac13597fea tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:53.520476 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f2248211-2aad-42b9-a97a-7b06c5da9ed1 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:53.520619 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-754d326e-8192-4c63-8ac6-68ac13597fea tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5652be64-c2a3-4b24-a3aa-c962f204c06e] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:53.521583 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f2248211-2aad-42b9-a97a-7b06c5da9ed1 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:53.522288 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f2248211-2aad-42b9-a97a-7b06c5da9ed1 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:53.528082 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-754d326e-8192-4c63-8ac6-68ac13597fea tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:53.528572 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-754d326e-8192-4c63-8ac6-68ac13597fea tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:53.529006 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-754d326e-8192-4c63-8ac6-68ac13597fea tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:53.530170 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-754d326e-8192-4c63-8ac6-68ac13597fea tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:53.530639 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-754d326e-8192-4c63-8ac6-68ac13597fea tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:53.531061 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-754d326e-8192-4c63-8ac6-68ac13597fea tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:53.753890 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-f2248211-2aad-42b9-a97a-7b06c5da9ed1 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:53.758358 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-754d326e-8192-4c63-8ac6-68ac13597fea tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:53.763095 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-f2248211-2aad-42b9-a97a-7b06c5da9ed1 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1364 microversion: 2.1 time: 0.268362 Oct 30 00:56:53.763692 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 95/183] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:53 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1364 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:53.770805 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-754d326e-8192-4c63-8ac6-68ac13597fea tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e" status: 200 len: 1557 microversion: 2.1 time: 0.255253 Oct 30 00:56:53.771555 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 89/184] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:53 2022] GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e => generated 1557 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:53.794090 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-2bb0f304-fecf-4979-a283-9c3af53d6936 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:53.794578 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-2bb0f304-fecf-4979-a283-9c3af53d6936 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5652be64-c2a3-4b24-a3aa-c962f204c06e] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:53.800430 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2bb0f304-fecf-4979-a283-9c3af53d6936 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:53.800899 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2bb0f304-fecf-4979-a283-9c3af53d6936 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:53.801316 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2bb0f304-fecf-4979-a283-9c3af53d6936 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:53.801984 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2bb0f304-fecf-4979-a283-9c3af53d6936 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:53.802419 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2bb0f304-fecf-4979-a283-9c3af53d6936 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:53.803838 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2bb0f304-fecf-4979-a283-9c3af53d6936 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:53.919814 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-2bb0f304-fecf-4979-a283-9c3af53d6936 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.129786 Oct 30 00:56:53.920423 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 96/185] 10.222.0.16 () {58 vars in 1275 bytes} [Sun Oct 30 00:56:53 2022] GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e/os-interface => generated 284 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:56:54.353197 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-29c09930-d181-4302-b8ff-6a52c63efda8 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:54.353831 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-29c09930-d181-4302-b8ff-6a52c63efda8 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:54.363224 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-29c09930-d181-4302-b8ff-6a52c63efda8 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:54.365172 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-29c09930-d181-4302-b8ff-6a52c63efda8 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:54.365794 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-29c09930-d181-4302-b8ff-6a52c63efda8 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:54.366847 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-29c09930-d181-4302-b8ff-6a52c63efda8 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:54.367449 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-29c09930-d181-4302-b8ff-6a52c63efda8 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:54.368041 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-29c09930-d181-4302-b8ff-6a52c63efda8 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:54.380719 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell1:SELECT=100 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:56:54.426040 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [req-695f18d9-d4ca-42dd-bcda-952ad6ae8840 req-dd8ec2b1-25ba-4f6b-b854-8be5932179ca service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "70ad3ec1-64bc-40ac-bb98-cc82d19e2a14", "name": "network-vif-plugged", "status": "completed", "tag": "dcb64cf9-6d5f-470b-8042-0cdbf3e91991"}]} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:56:54.443841 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-695f18d9-d4ca-42dd-bcda-952ad6ae8840 req-dd8ec2b1-25ba-4f6b-b854-8be5932179ca service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:54.444623 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-695f18d9-d4ca-42dd-bcda-952ad6ae8840 req-dd8ec2b1-25ba-4f6b-b854-8be5932179ca service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:54.445840 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-695f18d9-d4ca-42dd-bcda-952ad6ae8840 req-dd8ec2b1-25ba-4f6b-b854-8be5932179ca service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:54.462262 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.compute.server_external_events [req-695f18d9-d4ca-42dd-bcda-952ad6ae8840 req-dd8ec2b1-25ba-4f6b-b854-8be5932179ca service nova] Creating event network-vif-plugged:dcb64cf9-6d5f-470b-8042-0cdbf3e91991 for instance 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 on hv-cinder-maste Oct 30 00:56:54.469520 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [req-695f18d9-d4ca-42dd-bcda-952ad6ae8840 req-dd8ec2b1-25ba-4f6b-b854-8be5932179ca service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.046856 Oct 30 00:56:54.470341 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 97/186] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 00:56:54 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:56:54.547685 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-26fa3d81-8042-4853-9bb2-fa4acaeab164 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:54.548785 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-26fa3d81-8042-4853-9bb2-fa4acaeab164 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:54.570433 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-26fa3d81-8042-4853-9bb2-fa4acaeab164 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:54.571196 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-26fa3d81-8042-4853-9bb2-fa4acaeab164 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:54.572236 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-26fa3d81-8042-4853-9bb2-fa4acaeab164 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:54.573384 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-26fa3d81-8042-4853-9bb2-fa4acaeab164 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:54.574149 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-26fa3d81-8042-4853-9bb2-fa4acaeab164 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:54.574722 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-26fa3d81-8042-4853-9bb2-fa4acaeab164 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:54.976301 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-29c09930-d181-4302-b8ff-6a52c63efda8 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c92f7b4d09d74b8d93d739c1938694e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f7d505c7b9e4ca08d6437e6e529a039', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:54.985237 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-29c09930-d181-4302-b8ff-6a52c63efda8 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] 10.222.0.16 "GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1" status: 200 len: 1502 microversion: 2.1 time: 0.636475 Oct 30 00:56:54.992230 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 90/187] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:54 2022] GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1 => generated 1502 bytes in 644 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:54.997576 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-727020e5-64f7-4e0e-9568-3770891625af tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:54.998668 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-727020e5-64f7-4e0e-9568-3770891625af tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:55.017089 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-727020e5-64f7-4e0e-9568-3770891625af tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:55.017747 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-727020e5-64f7-4e0e-9568-3770891625af tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:55.022871 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-727020e5-64f7-4e0e-9568-3770891625af tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:55.023934 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-727020e5-64f7-4e0e-9568-3770891625af tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:55.024537 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-727020e5-64f7-4e0e-9568-3770891625af tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:55.026838 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-727020e5-64f7-4e0e-9568-3770891625af tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:55.107329 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-26fa3d81-8042-4853-9bb2-fa4acaeab164 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:55.119585 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-26fa3d81-8042-4853-9bb2-fa4acaeab164 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:55.123696 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-26fa3d81-8042-4853-9bb2-fa4acaeab164 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:55.126203 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-26fa3d81-8042-4853-9bb2-fa4acaeab164 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1406 microversion: 2.20 time: 0.586546 Oct 30 00:56:55.133350 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 98/188] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 00:56:54 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1406 bytes in 590 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 30 00:56:55.294767 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-727020e5-64f7-4e0e-9568-3770891625af tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:55.310055 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-727020e5-64f7-4e0e-9568-3770891625af tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1364 microversion: 2.1 time: 0.316650 Oct 30 00:56:55.310651 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 91/189] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:54 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1364 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:55.826960 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [req-092789c7-2be0-4599-87bb-c97c26aa7a29 req-40be7415-987b-49f1-acbe-8eba2c9abda8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5652be64-c2a3-4b24-a3aa-c962f204c06e", "tag": "9218be5d-9ff8-4e15-b073-e87bc5e0401c"}]} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:56:55.844673 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-092789c7-2be0-4599-87bb-c97c26aa7a29 req-40be7415-987b-49f1-acbe-8eba2c9abda8 service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:55.845216 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-092789c7-2be0-4599-87bb-c97c26aa7a29 req-40be7415-987b-49f1-acbe-8eba2c9abda8 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:55.846701 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-092789c7-2be0-4599-87bb-c97c26aa7a29 req-40be7415-987b-49f1-acbe-8eba2c9abda8 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:55.882298 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.compute.server_external_events [req-092789c7-2be0-4599-87bb-c97c26aa7a29 req-40be7415-987b-49f1-acbe-8eba2c9abda8 service nova] Creating event network-changed:9218be5d-9ff8-4e15-b073-e87bc5e0401c for instance 5652be64-c2a3-4b24-a3aa-c962f204c06e on hv-cinder-maste Oct 30 00:56:55.887570 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [req-092789c7-2be0-4599-87bb-c97c26aa7a29 req-40be7415-987b-49f1-acbe-8eba2c9abda8 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.064318 Oct 30 00:56:55.891305 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 99/190] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 00:56:55 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:56:56.152188 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-5ea2a94f-10f3-454f-8b66-4dde18b0c1af tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:56.153467 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-5ea2a94f-10f3-454f-8b66-4dde18b0c1af tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:56.160261 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5ea2a94f-10f3-454f-8b66-4dde18b0c1af tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:56.161179 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5ea2a94f-10f3-454f-8b66-4dde18b0c1af tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:56.161911 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5ea2a94f-10f3-454f-8b66-4dde18b0c1af tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:56.163176 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5ea2a94f-10f3-454f-8b66-4dde18b0c1af tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:56.163934 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5ea2a94f-10f3-454f-8b66-4dde18b0c1af tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:56.164518 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5ea2a94f-10f3-454f-8b66-4dde18b0c1af tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:56.305278 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-5ea2a94f-10f3-454f-8b66-4dde18b0c1af tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:56.314887 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-5ea2a94f-10f3-454f-8b66-4dde18b0c1af tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:56.316641 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-5ea2a94f-10f3-454f-8b66-4dde18b0c1af tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:56.317998 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-5ea2a94f-10f3-454f-8b66-4dde18b0c1af tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1598 microversion: 2.20 time: 0.170119 Oct 30 00:56:56.318765 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 92/191] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 00:56:56 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1598 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 00:56:56.336610 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-06664aa6-b6c7-42dd-b2d2-b33188cffcfd tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:56.337540 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-06664aa6-b6c7-42dd-b2d2-b33188cffcfd tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:56.344814 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-8935b45d-7a38-471b-b0bd-bdf590e3bdfa tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:56.345519 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-8935b45d-7a38-471b-b0bd-bdf590e3bdfa tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:56.352863 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-06664aa6-b6c7-42dd-b2d2-b33188cffcfd tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:56.353388 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-06664aa6-b6c7-42dd-b2d2-b33188cffcfd tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:56.353820 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-06664aa6-b6c7-42dd-b2d2-b33188cffcfd tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:56.354527 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-06664aa6-b6c7-42dd-b2d2-b33188cffcfd tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:56.354885 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8935b45d-7a38-471b-b0bd-bdf590e3bdfa tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:56.355025 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-06664aa6-b6c7-42dd-b2d2-b33188cffcfd tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:56.355370 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-06664aa6-b6c7-42dd-b2d2-b33188cffcfd tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:56.355498 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8935b45d-7a38-471b-b0bd-bdf590e3bdfa tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:56.356096 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8935b45d-7a38-471b-b0bd-bdf590e3bdfa tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:56.356925 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8935b45d-7a38-471b-b0bd-bdf590e3bdfa tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:56.357529 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8935b45d-7a38-471b-b0bd-bdf590e3bdfa tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:56.358081 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8935b45d-7a38-471b-b0bd-bdf590e3bdfa tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:56.512340 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-8935b45d-7a38-471b-b0bd-bdf590e3bdfa tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f/os-interface" status: 200 len: 285 microversion: 2.20 time: 0.172135 Oct 30 00:56:56.513109 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 93/192] 10.222.0.16 () {60 vars in 1316 bytes} [Sun Oct 30 00:56:56 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f/os-interface => generated 285 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 30 00:56:56.544483 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-06664aa6-b6c7-42dd-b2d2-b33188cffcfd tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:56.557032 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-06664aa6-b6c7-42dd-b2d2-b33188cffcfd tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1364 microversion: 2.1 time: 0.226798 Oct 30 00:56:56.557697 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 100/193] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:56 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1364 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:57.584589 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-d456c980-44e6-4ff8-90dc-374b0f8f8342 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:57.585852 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-d456c980-44e6-4ff8-90dc-374b0f8f8342 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:57.594658 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d456c980-44e6-4ff8-90dc-374b0f8f8342 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:57.595076 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d456c980-44e6-4ff8-90dc-374b0f8f8342 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:57.595492 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d456c980-44e6-4ff8-90dc-374b0f8f8342 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:57.596758 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d456c980-44e6-4ff8-90dc-374b0f8f8342 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:57.597208 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d456c980-44e6-4ff8-90dc-374b0f8f8342 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:57.597634 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d456c980-44e6-4ff8-90dc-374b0f8f8342 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:57.738369 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-d456c980-44e6-4ff8-90dc-374b0f8f8342 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:57.746389 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-d456c980-44e6-4ff8-90dc-374b0f8f8342 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1364 microversion: 2.1 time: 0.165912 Oct 30 00:56:57.746936 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 94/194] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:57 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1364 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:57.988423 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [req-efc0635d-da25-45ab-a453-90ff1ff9330d req-ef91e997-7396-4d9a-9a32-3ef7ec47636b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9491a8f4-2b76-4a4e-a6bd-6720114e482f", "tag": "5367850e-73d0-4814-af96-bd430c94c272"}]} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:56:57.998544 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-efc0635d-da25-45ab-a453-90ff1ff9330d req-ef91e997-7396-4d9a-9a32-3ef7ec47636b service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:57.999003 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-efc0635d-da25-45ab-a453-90ff1ff9330d req-ef91e997-7396-4d9a-9a32-3ef7ec47636b service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:57.999626 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-efc0635d-da25-45ab-a453-90ff1ff9330d req-ef91e997-7396-4d9a-9a32-3ef7ec47636b service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:58.015066 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.compute.server_external_events [req-efc0635d-da25-45ab-a453-90ff1ff9330d req-ef91e997-7396-4d9a-9a32-3ef7ec47636b service nova] Creating event network-changed:5367850e-73d0-4814-af96-bd430c94c272 for instance 9491a8f4-2b76-4a4e-a6bd-6720114e482f on hv-cinder-maste Oct 30 00:56:58.020901 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [req-efc0635d-da25-45ab-a453-90ff1ff9330d req-ef91e997-7396-4d9a-9a32-3ef7ec47636b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036475 Oct 30 00:56:58.022710 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 101/195] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 00:56:57 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:56:58.770355 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-b2ec2255-5c3b-4e25-8236-f3e7df856100 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:58.771078 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-b2ec2255-5c3b-4e25-8236-f3e7df856100 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:58.777350 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b2ec2255-5c3b-4e25-8236-f3e7df856100 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:58.777830 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b2ec2255-5c3b-4e25-8236-f3e7df856100 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:58.778254 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b2ec2255-5c3b-4e25-8236-f3e7df856100 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:58.778937 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b2ec2255-5c3b-4e25-8236-f3e7df856100 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:58.779383 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b2ec2255-5c3b-4e25-8236-f3e7df856100 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:58.779820 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b2ec2255-5c3b-4e25-8236-f3e7df856100 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:58.920602 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-b2ec2255-5c3b-4e25-8236-f3e7df856100 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:56:58.928512 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-b2ec2255-5c3b-4e25-8236-f3e7df856100 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1364 microversion: 2.1 time: 0.162099 Oct 30 00:56:58.929127 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 95/196] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:58 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1364 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:56:59.956605 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-c763e6b7-d587-40fb-a6d9-63b84d3a5091 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:56:59.957865 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-c763e6b7-d587-40fb-a6d9-63b84d3a5091 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:56:59.960997 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell0:SELECT=2 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:56:59.962039 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_api:SELECT=89 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:56:59.968269 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c763e6b7-d587-40fb-a6d9-63b84d3a5091 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:59.968738 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c763e6b7-d587-40fb-a6d9-63b84d3a5091 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:59.969157 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c763e6b7-d587-40fb-a6d9-63b84d3a5091 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:56:59.969837 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c763e6b7-d587-40fb-a6d9-63b84d3a5091 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:56:59.970275 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c763e6b7-d587-40fb-a6d9-63b84d3a5091 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:56:59.970682 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c763e6b7-d587-40fb-a6d9-63b84d3a5091 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:00.102821 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-c763e6b7-d587-40fb-a6d9-63b84d3a5091 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:57:00.112044 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-c763e6b7-d587-40fb-a6d9-63b84d3a5091 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1364 microversion: 2.1 time: 0.162495 Oct 30 00:57:00.112750 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 102/197] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:56:59 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1364 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:57:01.137317 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-d0eb06b9-2460-4ebd-b034-3c88071f9fda tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:57:01.138784 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-d0eb06b9-2460-4ebd-b034-3c88071f9fda tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:57:01.145611 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d0eb06b9-2460-4ebd-b034-3c88071f9fda tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:01.146088 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d0eb06b9-2460-4ebd-b034-3c88071f9fda tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:01.146513 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d0eb06b9-2460-4ebd-b034-3c88071f9fda tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:01.147203 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d0eb06b9-2460-4ebd-b034-3c88071f9fda tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:01.147638 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d0eb06b9-2460-4ebd-b034-3c88071f9fda tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:01.148076 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d0eb06b9-2460-4ebd-b034-3c88071f9fda tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:01.295372 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-d0eb06b9-2460-4ebd-b034-3c88071f9fda tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:57:01.308212 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-d0eb06b9-2460-4ebd-b034-3c88071f9fda tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1364 microversion: 2.1 time: 0.174719 Oct 30 00:57:01.310330 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 96/198] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:57:01 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1364 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:57:01.926385 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-0cba796a-6af9-4fbd-8f2c-6f460be1edcf tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "9352799c-08be-4964-9dba-b6419a66165f"}} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:57:01.928865 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-0cba796a-6af9-4fbd-8f2c-6f460be1edcf tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:57:01.935142 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0cba796a-6af9-4fbd-8f2c-6f460be1edcf tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:01.935621 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0cba796a-6af9-4fbd-8f2c-6f460be1edcf tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:01.936103 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0cba796a-6af9-4fbd-8f2c-6f460be1edcf tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:01.936819 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0cba796a-6af9-4fbd-8f2c-6f460be1edcf tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:01.937260 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0cba796a-6af9-4fbd-8f2c-6f460be1edcf tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:01.937727 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0cba796a-6af9-4fbd-8f2c-6f460be1edcf tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:02.020726 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell1:SELECT=100 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:57:02.334726 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-b1f6a966-96a0-4634-9dd3-ea7d43b73805 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:57:02.335398 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-b1f6a966-96a0-4634-9dd3-ea7d43b73805 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:57:02.346143 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b1f6a966-96a0-4634-9dd3-ea7d43b73805 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:02.346653 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b1f6a966-96a0-4634-9dd3-ea7d43b73805 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:02.347090 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b1f6a966-96a0-4634-9dd3-ea7d43b73805 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:02.350224 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b1f6a966-96a0-4634-9dd3-ea7d43b73805 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:02.350394 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b1f6a966-96a0-4634-9dd3-ea7d43b73805 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:02.350840 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b1f6a966-96a0-4634-9dd3-ea7d43b73805 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:02.495420 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-b1f6a966-96a0-4634-9dd3-ea7d43b73805 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:57:02.503309 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-b1f6a966-96a0-4634-9dd3-ea7d43b73805 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1364 microversion: 2.1 time: 0.173124 Oct 30 00:57:02.504025 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 97/199] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:57:02 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1364 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:57:03.526180 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-8f4fa79d-68bf-42c8-8f4c-739a47b0530b tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:57:03.526848 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-8f4fa79d-68bf-42c8-8f4c-739a47b0530b tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:57:03.533559 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8f4fa79d-68bf-42c8-8f4c-739a47b0530b tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:03.533946 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8f4fa79d-68bf-42c8-8f4c-739a47b0530b tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:03.534379 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8f4fa79d-68bf-42c8-8f4c-739a47b0530b tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:03.535068 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8f4fa79d-68bf-42c8-8f4c-739a47b0530b tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:03.535515 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8f4fa79d-68bf-42c8-8f4c-739a47b0530b tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:03.535964 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8f4fa79d-68bf-42c8-8f4c-739a47b0530b tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:03.673452 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-8f4fa79d-68bf-42c8-8f4c-739a47b0530b tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:57:03.681734 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-8f4fa79d-68bf-42c8-8f4c-739a47b0530b tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1364 microversion: 2.1 time: 0.159588 Oct 30 00:57:03.682328 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 98/200] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:57:03 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1364 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:57:04.704954 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-d72e8ad3-8852-4876-ac89-5a29eb97798f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:57:04.705559 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-d72e8ad3-8852-4876-ac89-5a29eb97798f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:57:04.711942 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d72e8ad3-8852-4876-ac89-5a29eb97798f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:04.712487 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d72e8ad3-8852-4876-ac89-5a29eb97798f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:04.712917 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d72e8ad3-8852-4876-ac89-5a29eb97798f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:04.713636 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d72e8ad3-8852-4876-ac89-5a29eb97798f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:04.714087 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d72e8ad3-8852-4876-ac89-5a29eb97798f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:04.714503 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d72e8ad3-8852-4876-ac89-5a29eb97798f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:04.855092 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-d72e8ad3-8852-4876-ac89-5a29eb97798f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:57:04.863354 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-d72e8ad3-8852-4876-ac89-5a29eb97798f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1364 microversion: 2.1 time: 0.162448 Oct 30 00:57:04.863934 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 99/201] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:57:04 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1364 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:57:05.886796 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-21c8cf6d-a272-4c8a-816d-8dd38c6a1d50 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:57:05.887437 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-21c8cf6d-a272-4c8a-816d-8dd38c6a1d50 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:57:05.893687 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-21c8cf6d-a272-4c8a-816d-8dd38c6a1d50 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:05.894169 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-21c8cf6d-a272-4c8a-816d-8dd38c6a1d50 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:05.894721 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-21c8cf6d-a272-4c8a-816d-8dd38c6a1d50 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:05.895589 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-21c8cf6d-a272-4c8a-816d-8dd38c6a1d50 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:05.896089 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-21c8cf6d-a272-4c8a-816d-8dd38c6a1d50 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:05.896510 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-21c8cf6d-a272-4c8a-816d-8dd38c6a1d50 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:06.215927 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-21c8cf6d-a272-4c8a-816d-8dd38c6a1d50 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:57:06.224258 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-21c8cf6d-a272-4c8a-816d-8dd38c6a1d50 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1364 microversion: 2.1 time: 0.341436 Oct 30 00:57:06.224834 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 100/202] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:57:05 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1364 bytes in 343 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:57:07.248917 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-4ecf1ead-c70f-4975-b89c-95d7ba521097 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:57:07.249539 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-4ecf1ead-c70f-4975-b89c-95d7ba521097 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:57:07.255962 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4ecf1ead-c70f-4975-b89c-95d7ba521097 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:07.256446 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4ecf1ead-c70f-4975-b89c-95d7ba521097 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:07.256875 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4ecf1ead-c70f-4975-b89c-95d7ba521097 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:07.257558 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4ecf1ead-c70f-4975-b89c-95d7ba521097 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:07.258007 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4ecf1ead-c70f-4975-b89c-95d7ba521097 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:07.258424 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4ecf1ead-c70f-4975-b89c-95d7ba521097 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:07.393789 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-4ecf1ead-c70f-4975-b89c-95d7ba521097 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:57:07.401847 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-4ecf1ead-c70f-4975-b89c-95d7ba521097 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1364 microversion: 2.1 time: 0.156772 Oct 30 00:57:07.402536 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 101/203] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:57:07 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1364 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:57:08.428686 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-a175c436-d903-4664-93ed-8a5f91612ba6 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:57:08.429624 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-a175c436-d903-4664-93ed-8a5f91612ba6 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:57:08.437378 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a175c436-d903-4664-93ed-8a5f91612ba6 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:08.437851 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a175c436-d903-4664-93ed-8a5f91612ba6 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:08.438365 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a175c436-d903-4664-93ed-8a5f91612ba6 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:08.439075 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a175c436-d903-4664-93ed-8a5f91612ba6 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:08.439588 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a175c436-d903-4664-93ed-8a5f91612ba6 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:08.440104 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a175c436-d903-4664-93ed-8a5f91612ba6 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:08.572860 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-a175c436-d903-4664-93ed-8a5f91612ba6 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:57:08.584670 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-a175c436-d903-4664-93ed-8a5f91612ba6 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1364 microversion: 2.1 time: 0.162480 Oct 30 00:57:08.585276 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 102/204] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:57:08 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1364 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:57:09.608239 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-1b5af99f-ccfc-4d5c-b077-ce19c48db085 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:57:09.608816 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-1b5af99f-ccfc-4d5c-b077-ce19c48db085 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:57:09.615269 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1b5af99f-ccfc-4d5c-b077-ce19c48db085 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:09.615747 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1b5af99f-ccfc-4d5c-b077-ce19c48db085 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:09.616206 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1b5af99f-ccfc-4d5c-b077-ce19c48db085 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:09.616893 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1b5af99f-ccfc-4d5c-b077-ce19c48db085 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:09.617343 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1b5af99f-ccfc-4d5c-b077-ce19c48db085 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:09.617782 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1b5af99f-ccfc-4d5c-b077-ce19c48db085 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:09.746203 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-1b5af99f-ccfc-4d5c-b077-ce19c48db085 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:57:09.753789 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-1b5af99f-ccfc-4d5c-b077-ce19c48db085 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1364 microversion: 2.1 time: 0.149549 Oct 30 00:57:09.754377 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 103/205] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:57:09 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1364 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:57:10.779036 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-89587d6a-d8c8-44a0-a9bd-6aa88efea30d tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:57:10.779653 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-89587d6a-d8c8-44a0-a9bd-6aa88efea30d tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:57:10.785746 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-89587d6a-d8c8-44a0-a9bd-6aa88efea30d tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:10.786228 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-89587d6a-d8c8-44a0-a9bd-6aa88efea30d tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:10.786660 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-89587d6a-d8c8-44a0-a9bd-6aa88efea30d tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:10.787372 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-89587d6a-d8c8-44a0-a9bd-6aa88efea30d tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:10.787852 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-89587d6a-d8c8-44a0-a9bd-6aa88efea30d tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:10.788270 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-89587d6a-d8c8-44a0-a9bd-6aa88efea30d tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:10.917742 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-89587d6a-d8c8-44a0-a9bd-6aa88efea30d tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:57:10.925727 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-89587d6a-d8c8-44a0-a9bd-6aa88efea30d tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1364 microversion: 2.1 time: 0.150889 Oct 30 00:57:10.926342 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 104/206] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:57:10 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1364 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:57:11.933620 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_api:SELECT=3 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:57:11.949573 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-3e7f1f1b-8fcb-4d7f-b3eb-0173f9793379 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:57:11.950254 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-3e7f1f1b-8fcb-4d7f-b3eb-0173f9793379 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:57:11.956372 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3e7f1f1b-8fcb-4d7f-b3eb-0173f9793379 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:11.957052 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3e7f1f1b-8fcb-4d7f-b3eb-0173f9793379 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:11.957708 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3e7f1f1b-8fcb-4d7f-b3eb-0173f9793379 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:11.958364 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3e7f1f1b-8fcb-4d7f-b3eb-0173f9793379 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:11.958811 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3e7f1f1b-8fcb-4d7f-b3eb-0173f9793379 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:11.959227 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3e7f1f1b-8fcb-4d7f-b3eb-0173f9793379 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:12.097597 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-3e7f1f1b-8fcb-4d7f-b3eb-0173f9793379 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:57:12.105378 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-3e7f1f1b-8fcb-4d7f-b3eb-0173f9793379 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1364 microversion: 2.1 time: 0.159967 Oct 30 00:57:12.105879 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 105/207] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:57:11 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1364 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:57:13.132456 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-35cbd0e2-c69c-47eb-b4eb-55b90d0ecf93 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:57:13.133564 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-35cbd0e2-c69c-47eb-b4eb-55b90d0ecf93 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:57:13.142308 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-35cbd0e2-c69c-47eb-b4eb-55b90d0ecf93 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:13.142804 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-35cbd0e2-c69c-47eb-b4eb-55b90d0ecf93 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:13.143297 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-35cbd0e2-c69c-47eb-b4eb-55b90d0ecf93 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:13.144422 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-35cbd0e2-c69c-47eb-b4eb-55b90d0ecf93 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:13.145031 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-35cbd0e2-c69c-47eb-b4eb-55b90d0ecf93 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:13.145618 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-35cbd0e2-c69c-47eb-b4eb-55b90d0ecf93 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:13.281740 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-35cbd0e2-c69c-47eb-b4eb-55b90d0ecf93 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:57:13.289859 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-35cbd0e2-c69c-47eb-b4eb-55b90d0ecf93 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1364 microversion: 2.1 time: 0.163456 Oct 30 00:57:13.290434 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 106/208] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:57:13 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1364 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:57:14.314408 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-2943e417-6786-4614-be1f-79381840288a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:57:14.315073 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-2943e417-6786-4614-be1f-79381840288a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:57:14.321972 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2943e417-6786-4614-be1f-79381840288a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:14.322451 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2943e417-6786-4614-be1f-79381840288a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:14.322878 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2943e417-6786-4614-be1f-79381840288a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:14.323573 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2943e417-6786-4614-be1f-79381840288a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:14.324069 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2943e417-6786-4614-be1f-79381840288a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:14.324485 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2943e417-6786-4614-be1f-79381840288a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:14.328516 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell1:SELECT=100 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:57:14.464526 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-2943e417-6786-4614-be1f-79381840288a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:57:14.473114 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-2943e417-6786-4614-be1f-79381840288a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1364 microversion: 2.1 time: 0.163525 Oct 30 00:57:14.473723 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 107/209] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:57:14 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1364 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:57:15.500712 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-c2ee1664-d52d-4212-9e81-de570fd78967 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:57:15.501793 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-c2ee1664-d52d-4212-9e81-de570fd78967 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:57:15.511776 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c2ee1664-d52d-4212-9e81-de570fd78967 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:15.512618 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c2ee1664-d52d-4212-9e81-de570fd78967 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:15.513337 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c2ee1664-d52d-4212-9e81-de570fd78967 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:15.514466 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c2ee1664-d52d-4212-9e81-de570fd78967 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:15.515215 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c2ee1664-d52d-4212-9e81-de570fd78967 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:15.515850 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c2ee1664-d52d-4212-9e81-de570fd78967 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:15.646830 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-c2ee1664-d52d-4212-9e81-de570fd78967 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:57:15.655225 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-c2ee1664-d52d-4212-9e81-de570fd78967 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1364 microversion: 2.1 time: 0.161270 Oct 30 00:57:15.655849 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 108/210] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:57:15 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1364 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:57:16.679177 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-06134fb2-a593-45e1-8e3c-32421de6242a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:57:16.679861 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-06134fb2-a593-45e1-8e3c-32421de6242a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:57:16.686641 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-06134fb2-a593-45e1-8e3c-32421de6242a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:16.687132 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-06134fb2-a593-45e1-8e3c-32421de6242a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:16.687561 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-06134fb2-a593-45e1-8e3c-32421de6242a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:16.688393 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-06134fb2-a593-45e1-8e3c-32421de6242a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:16.688961 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-06134fb2-a593-45e1-8e3c-32421de6242a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:16.689407 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-06134fb2-a593-45e1-8e3c-32421de6242a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:16.842378 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-06134fb2-a593-45e1-8e3c-32421de6242a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:57:16.850480 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-06134fb2-a593-45e1-8e3c-32421de6242a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1364 microversion: 2.1 time: 0.175861 Oct 30 00:57:16.851051 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 109/211] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:57:16 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1364 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:57:17.880121 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-152a48bd-88b3-49e6-a10e-f446bd47e810 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:57:17.881164 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-152a48bd-88b3-49e6-a10e-f446bd47e810 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:57:17.891157 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-152a48bd-88b3-49e6-a10e-f446bd47e810 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:17.891965 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-152a48bd-88b3-49e6-a10e-f446bd47e810 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:17.892728 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-152a48bd-88b3-49e6-a10e-f446bd47e810 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:17.893629 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-152a48bd-88b3-49e6-a10e-f446bd47e810 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:17.894081 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-152a48bd-88b3-49e6-a10e-f446bd47e810 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:17.894498 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-152a48bd-88b3-49e6-a10e-f446bd47e810 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:18.033234 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-152a48bd-88b3-49e6-a10e-f446bd47e810 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:57:18.043334 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-152a48bd-88b3-49e6-a10e-f446bd47e810 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1364 microversion: 2.1 time: 0.169687 Oct 30 00:57:18.044186 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 110/212] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:57:17 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1364 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:57:18.711001 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-0cba796a-6af9-4fbd-8f2c-6f460be1edcf tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] 10.222.0.16 "POST /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 16.788719 Oct 30 00:57:18.712095 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 103/213] 10.222.0.16 () {60 vars in 1323 bytes} [Sun Oct 30 00:57:01 2022] POST /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1/os-volume_attachments => generated 194 bytes in 16790 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:57:19.068566 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-bd4390f3-ad0b-4f6d-942c-84a64f2af9c3 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:57:19.069154 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-bd4390f3-ad0b-4f6d-942c-84a64f2af9c3 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:57:19.075837 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bd4390f3-ad0b-4f6d-942c-84a64f2af9c3 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:19.076330 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bd4390f3-ad0b-4f6d-942c-84a64f2af9c3 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:19.076745 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bd4390f3-ad0b-4f6d-942c-84a64f2af9c3 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:19.077429 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bd4390f3-ad0b-4f6d-942c-84a64f2af9c3 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:19.077880 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bd4390f3-ad0b-4f6d-942c-84a64f2af9c3 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:19.078298 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bd4390f3-ad0b-4f6d-942c-84a64f2af9c3 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:19.218609 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-bd4390f3-ad0b-4f6d-942c-84a64f2af9c3 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:57:19.232926 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-bd4390f3-ad0b-4f6d-942c-84a64f2af9c3 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1364 microversion: 2.1 time: 0.168848 Oct 30 00:57:19.233623 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 111/214] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:57:19 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1364 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:57:20.260412 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-bdc5f6ae-beb4-4134-9401-0c3cbd5d359a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:57:20.261165 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-bdc5f6ae-beb4-4134-9401-0c3cbd5d359a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:57:20.268169 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bdc5f6ae-beb4-4134-9401-0c3cbd5d359a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:20.268650 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bdc5f6ae-beb4-4134-9401-0c3cbd5d359a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:20.269082 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bdc5f6ae-beb4-4134-9401-0c3cbd5d359a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:20.270233 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bdc5f6ae-beb4-4134-9401-0c3cbd5d359a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:20.270710 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bdc5f6ae-beb4-4134-9401-0c3cbd5d359a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:20.271134 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bdc5f6ae-beb4-4134-9401-0c3cbd5d359a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:20.414655 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-bdc5f6ae-beb4-4134-9401-0c3cbd5d359a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:57:20.423031 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-bdc5f6ae-beb4-4134-9401-0c3cbd5d359a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1550 microversion: 2.1 time: 0.167326 Oct 30 00:57:20.423541 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 104/215] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:57:20 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1550 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:57:20.446554 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-4b408cf1-29e0-499a-b7e8-2883c8a36d15 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:57:20.447102 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-4b408cf1-29e0-499a-b7e8-2883c8a36d15 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:57:20.448970 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_api:SELECT=62 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:57:20.454315 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4b408cf1-29e0-499a-b7e8-2883c8a36d15 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:20.454792 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4b408cf1-29e0-499a-b7e8-2883c8a36d15 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:20.455217 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4b408cf1-29e0-499a-b7e8-2883c8a36d15 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:20.455942 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4b408cf1-29e0-499a-b7e8-2883c8a36d15 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:20.456391 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4b408cf1-29e0-499a-b7e8-2883c8a36d15 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:20.456814 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4b408cf1-29e0-499a-b7e8-2883c8a36d15 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:20.537626 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-4b408cf1-29e0-499a-b7e8-2883c8a36d15 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.095389 Oct 30 00:57:20.538235 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 112/216] 10.222.0.16 () {58 vars in 1275 bytes} [Sun Oct 30 00:57:20 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14/os-interface => generated 284 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:57:21.507951 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [req-e1b48642-8824-4c60-9439-5c7d146b156d req-f50ffa44-3c86-43fe-9982-832888b2e825 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "70ad3ec1-64bc-40ac-bb98-cc82d19e2a14", "tag": "dcb64cf9-6d5f-470b-8042-0cdbf3e91991"}]} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:57:21.518254 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-e1b48642-8824-4c60-9439-5c7d146b156d req-f50ffa44-3c86-43fe-9982-832888b2e825 service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:21.518706 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-e1b48642-8824-4c60-9439-5c7d146b156d req-f50ffa44-3c86-43fe-9982-832888b2e825 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:21.519172 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-e1b48642-8824-4c60-9439-5c7d146b156d req-f50ffa44-3c86-43fe-9982-832888b2e825 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:21.531223 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.compute.server_external_events [req-e1b48642-8824-4c60-9439-5c7d146b156d req-f50ffa44-3c86-43fe-9982-832888b2e825 service nova] Creating event network-changed:dcb64cf9-6d5f-470b-8042-0cdbf3e91991 for instance 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 on hv-cinder-maste Oct 30 00:57:21.538835 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [req-e1b48642-8824-4c60-9439-5c7d146b156d req-f50ffa44-3c86-43fe-9982-832888b2e825 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034953 Oct 30 00:57:21.539763 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 105/217] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 00:57:21 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:57:49.800551 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_api:SELECT=1 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:57:49.801302 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell1:SELECT=32 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:57:49.807044 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-c66f7227-f898-4694-bc9d-2d2733f6d039 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:57:49.807537 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-c66f7227-f898-4694-bc9d-2d2733f6d039 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:57:49.814481 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c66f7227-f898-4694-bc9d-2d2733f6d039 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:49.814953 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c66f7227-f898-4694-bc9d-2d2733f6d039 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:49.815373 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c66f7227-f898-4694-bc9d-2d2733f6d039 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:49.816080 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c66f7227-f898-4694-bc9d-2d2733f6d039 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:49.816524 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c66f7227-f898-4694-bc9d-2d2733f6d039 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:49.816968 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c66f7227-f898-4694-bc9d-2d2733f6d039 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:49.841735 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-c66f7227-f898-4694-bc9d-2d2733f6d039 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] Going to try to terminate instance {{(pid=118961) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 30 00:57:49.940543 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-c66f7227-f898-4694-bc9d-2d2733f6d039 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] 10.222.0.16 "DELETE /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1" status: 204 len: 0 microversion: 2.1 time: 0.140878 Oct 30 00:57:49.941142 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 113/218] 10.222.0.16 () {58 vars in 1239 bytes} [Sun Oct 30 00:57:49 2022] DELETE /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Oct 30 00:57:49.949020 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_api:SELECT=4 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:57:49.950308 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell1:SELECT=13,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:57:49.955313 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-cd2bc2da-7a9b-472c-8f1e-749233e38ca2 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:57:49.956116 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-cd2bc2da-7a9b-472c-8f1e-749233e38ca2 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:57:49.963795 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-cd2bc2da-7a9b-472c-8f1e-749233e38ca2 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:49.964347 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-cd2bc2da-7a9b-472c-8f1e-749233e38ca2 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:49.964893 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-cd2bc2da-7a9b-472c-8f1e-749233e38ca2 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:49.965646 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-cd2bc2da-7a9b-472c-8f1e-749233e38ca2 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:49.966156 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-cd2bc2da-7a9b-472c-8f1e-749233e38ca2 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:49.966634 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-cd2bc2da-7a9b-472c-8f1e-749233e38ca2 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:50.123011 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-cd2bc2da-7a9b-472c-8f1e-749233e38ca2 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c92f7b4d09d74b8d93d739c1938694e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f7d505c7b9e4ca08d6437e6e529a039', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:57:50.132939 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-cd2bc2da-7a9b-472c-8f1e-749233e38ca2 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] 10.222.0.16 "GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1" status: 200 len: 1554 microversion: 2.1 time: 0.186392 Oct 30 00:57:50.136084 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 106/219] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:57:49 2022] GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1 => generated 1554 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:57:51.159372 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-7240ff8f-3d19-49a5-a3d7-9dbe653dc80c tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:57:51.160096 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-7240ff8f-3d19-49a5-a3d7-9dbe653dc80c tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:57:51.166337 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7240ff8f-3d19-49a5-a3d7-9dbe653dc80c tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:51.166809 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7240ff8f-3d19-49a5-a3d7-9dbe653dc80c tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:51.167238 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7240ff8f-3d19-49a5-a3d7-9dbe653dc80c tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:51.167949 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7240ff8f-3d19-49a5-a3d7-9dbe653dc80c tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:51.168388 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7240ff8f-3d19-49a5-a3d7-9dbe653dc80c tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:51.168799 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7240ff8f-3d19-49a5-a3d7-9dbe653dc80c tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:51.307058 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-7240ff8f-3d19-49a5-a3d7-9dbe653dc80c tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c92f7b4d09d74b8d93d739c1938694e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f7d505c7b9e4ca08d6437e6e529a039', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:57:51.315512 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-7240ff8f-3d19-49a5-a3d7-9dbe653dc80c tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] 10.222.0.16 "GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1" status: 200 len: 1554 microversion: 2.1 time: 0.160453 Oct 30 00:57:51.316057 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 114/220] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:57:51 2022] GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1 => generated 1554 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:57:52.338478 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-f24458dc-9029-478a-a75b-64786ec5a20b tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:57:52.339647 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-f24458dc-9029-478a-a75b-64786ec5a20b tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:57:52.346292 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f24458dc-9029-478a-a75b-64786ec5a20b tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:52.346807 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f24458dc-9029-478a-a75b-64786ec5a20b tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:52.347642 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f24458dc-9029-478a-a75b-64786ec5a20b tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:52.348630 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f24458dc-9029-478a-a75b-64786ec5a20b tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:52.349196 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f24458dc-9029-478a-a75b-64786ec5a20b tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:52.349672 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f24458dc-9029-478a-a75b-64786ec5a20b tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:52.485560 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-f24458dc-9029-478a-a75b-64786ec5a20b tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c92f7b4d09d74b8d93d739c1938694e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f7d505c7b9e4ca08d6437e6e529a039', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:57:52.495002 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-f24458dc-9029-478a-a75b-64786ec5a20b tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] 10.222.0.16 "GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1" status: 200 len: 1554 microversion: 2.1 time: 0.160354 Oct 30 00:57:52.495586 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 107/221] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:57:52 2022] GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1 => generated 1554 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:57:53.527752 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-cbf84d59-dd67-4cb2-9508-357747468c74 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:57:53.528797 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-cbf84d59-dd67-4cb2-9508-357747468c74 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:57:53.538119 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cbf84d59-dd67-4cb2-9508-357747468c74 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:53.538629 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cbf84d59-dd67-4cb2-9508-357747468c74 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:53.539054 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cbf84d59-dd67-4cb2-9508-357747468c74 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:53.539753 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cbf84d59-dd67-4cb2-9508-357747468c74 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:53.540213 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cbf84d59-dd67-4cb2-9508-357747468c74 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:53.540627 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cbf84d59-dd67-4cb2-9508-357747468c74 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:53.680165 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-cbf84d59-dd67-4cb2-9508-357747468c74 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c92f7b4d09d74b8d93d739c1938694e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f7d505c7b9e4ca08d6437e6e529a039', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:57:53.692822 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-cbf84d59-dd67-4cb2-9508-357747468c74 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] 10.222.0.16 "GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1" status: 200 len: 1554 microversion: 2.1 time: 0.171766 Oct 30 00:57:53.693401 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 115/222] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:57:53 2022] GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1 => generated 1554 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:57:54.720820 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-efa6ed7b-5925-41a3-bf19-1fa6e890bff0 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:57:54.721826 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-efa6ed7b-5925-41a3-bf19-1fa6e890bff0 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:57:54.733080 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-efa6ed7b-5925-41a3-bf19-1fa6e890bff0 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:54.733906 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-efa6ed7b-5925-41a3-bf19-1fa6e890bff0 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:54.734626 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-efa6ed7b-5925-41a3-bf19-1fa6e890bff0 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:54.735778 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-efa6ed7b-5925-41a3-bf19-1fa6e890bff0 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:54.736560 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-efa6ed7b-5925-41a3-bf19-1fa6e890bff0 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:54.737255 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-efa6ed7b-5925-41a3-bf19-1fa6e890bff0 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:54.872844 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-efa6ed7b-5925-41a3-bf19-1fa6e890bff0 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c92f7b4d09d74b8d93d739c1938694e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f7d505c7b9e4ca08d6437e6e529a039', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:57:54.882007 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-efa6ed7b-5925-41a3-bf19-1fa6e890bff0 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] 10.222.0.16 "GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1" status: 200 len: 1554 microversion: 2.1 time: 0.168159 Oct 30 00:57:54.882584 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 108/223] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:57:54 2022] GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1 => generated 1554 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:57:55.905970 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-03a646d3-0f96-4bed-9f24-cb6087731ac8 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:57:55.906501 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-03a646d3-0f96-4bed-9f24-cb6087731ac8 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:57:55.912828 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-03a646d3-0f96-4bed-9f24-cb6087731ac8 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:55.913333 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-03a646d3-0f96-4bed-9f24-cb6087731ac8 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:55.913759 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-03a646d3-0f96-4bed-9f24-cb6087731ac8 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:55.914445 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-03a646d3-0f96-4bed-9f24-cb6087731ac8 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:55.914888 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-03a646d3-0f96-4bed-9f24-cb6087731ac8 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:55.915297 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-03a646d3-0f96-4bed-9f24-cb6087731ac8 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:56.189082 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-03a646d3-0f96-4bed-9f24-cb6087731ac8 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c92f7b4d09d74b8d93d739c1938694e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f7d505c7b9e4ca08d6437e6e529a039', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:57:56.202050 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-03a646d3-0f96-4bed-9f24-cb6087731ac8 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] 10.222.0.16 "GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1" status: 200 len: 1554 microversion: 2.1 time: 0.299809 Oct 30 00:57:56.202843 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 116/224] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:57:55 2022] GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1 => generated 1554 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:57:57.231432 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-f830df33-fac8-4dbf-b1b1-cbae4cfdb57c tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:57:57.232520 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-f830df33-fac8-4dbf-b1b1-cbae4cfdb57c tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:57:57.243190 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f830df33-fac8-4dbf-b1b1-cbae4cfdb57c tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:57.244083 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f830df33-fac8-4dbf-b1b1-cbae4cfdb57c tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:57.244794 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f830df33-fac8-4dbf-b1b1-cbae4cfdb57c tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:57.245918 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f830df33-fac8-4dbf-b1b1-cbae4cfdb57c tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:57.246647 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f830df33-fac8-4dbf-b1b1-cbae4cfdb57c tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:57.247344 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f830df33-fac8-4dbf-b1b1-cbae4cfdb57c tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:57.387240 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-f830df33-fac8-4dbf-b1b1-cbae4cfdb57c tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c92f7b4d09d74b8d93d739c1938694e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f7d505c7b9e4ca08d6437e6e529a039', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:57:57.399911 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-f830df33-fac8-4dbf-b1b1-cbae4cfdb57c tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] 10.222.0.16 "GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1" status: 200 len: 1554 microversion: 2.1 time: 0.174968 Oct 30 00:57:57.400617 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 109/225] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:57:57 2022] GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1 => generated 1554 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:57:58.427440 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-0bcee0f7-a5f9-4178-b2a0-ec9c31df67ac tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:57:58.428040 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-0bcee0f7-a5f9-4178-b2a0-ec9c31df67ac tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:57:58.434952 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0bcee0f7-a5f9-4178-b2a0-ec9c31df67ac tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:58.435413 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0bcee0f7-a5f9-4178-b2a0-ec9c31df67ac tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:58.435888 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0bcee0f7-a5f9-4178-b2a0-ec9c31df67ac tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:58.436582 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0bcee0f7-a5f9-4178-b2a0-ec9c31df67ac tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:58.437033 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0bcee0f7-a5f9-4178-b2a0-ec9c31df67ac tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:58.437454 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0bcee0f7-a5f9-4178-b2a0-ec9c31df67ac tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:58.584138 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-0bcee0f7-a5f9-4178-b2a0-ec9c31df67ac tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c92f7b4d09d74b8d93d739c1938694e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f7d505c7b9e4ca08d6437e6e529a039', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:57:58.593377 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-0bcee0f7-a5f9-4178-b2a0-ec9c31df67ac tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] 10.222.0.16 "GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1" status: 200 len: 1554 microversion: 2.1 time: 0.169942 Oct 30 00:57:58.593951 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 117/226] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:57:58 2022] GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1 => generated 1554 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:57:59.267241 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [req-e14d39b9-a759-4dd2-91ca-1eda8eea0f2a req-b64653ee-02ee-4623-8a8c-1ed14299f048 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0f858c8d-6414-403e-b8d3-3ecd728e84a1", "name": "network-vif-deleted", "tag": "2c0a4a56-006d-4324-b26e-f48bac7f7a5b"}]} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:57:59.278064 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-e14d39b9-a759-4dd2-91ca-1eda8eea0f2a req-b64653ee-02ee-4623-8a8c-1ed14299f048 service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:59.278527 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-e14d39b9-a759-4dd2-91ca-1eda8eea0f2a req-b64653ee-02ee-4623-8a8c-1ed14299f048 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:59.279112 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-e14d39b9-a759-4dd2-91ca-1eda8eea0f2a req-b64653ee-02ee-4623-8a8c-1ed14299f048 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:59.293617 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.compute.server_external_events [req-e14d39b9-a759-4dd2-91ca-1eda8eea0f2a req-b64653ee-02ee-4623-8a8c-1ed14299f048 service nova] Creating event network-vif-deleted:2c0a4a56-006d-4324-b26e-f48bac7f7a5b for instance 0f858c8d-6414-403e-b8d3-3ecd728e84a1 on hv-cinder-maste Oct 30 00:57:59.309649 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [req-e14d39b9-a759-4dd2-91ca-1eda8eea0f2a req-b64653ee-02ee-4623-8a8c-1ed14299f048 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.045872 Oct 30 00:57:59.310719 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 110/227] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 00:57:59 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:57:59.615755 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-29098a2c-2310-4d51-8ab7-c4b27dd24c23 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:57:59.616363 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-29098a2c-2310-4d51-8ab7-c4b27dd24c23 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:57:59.623158 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-29098a2c-2310-4d51-8ab7-c4b27dd24c23 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:59.623696 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-29098a2c-2310-4d51-8ab7-c4b27dd24c23 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:59.624072 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-29098a2c-2310-4d51-8ab7-c4b27dd24c23 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:59.624959 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-29098a2c-2310-4d51-8ab7-c4b27dd24c23 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:57:59.625415 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-29098a2c-2310-4d51-8ab7-c4b27dd24c23 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:57:59.625790 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-29098a2c-2310-4d51-8ab7-c4b27dd24c23 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:57:59.687204 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-29098a2c-2310-4d51-8ab7-c4b27dd24c23 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c92f7b4d09d74b8d93d739c1938694e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f7d505c7b9e4ca08d6437e6e529a039', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:57:59.695850 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-29098a2c-2310-4d51-8ab7-c4b27dd24c23 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] 10.222.0.16 "GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1" status: 200 len: 1345 microversion: 2.1 time: 0.084023 Oct 30 00:57:59.696410 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 118/228] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:57:59 2022] GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1 => generated 1345 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:00.720305 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-8c8e4509-c0e9-4d9e-8101-e73655e83f51 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:00.720998 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-8c8e4509-c0e9-4d9e-8101-e73655e83f51 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:00.728042 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8c8e4509-c0e9-4d9e-8101-e73655e83f51 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:00.728537 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8c8e4509-c0e9-4d9e-8101-e73655e83f51 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:00.728962 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8c8e4509-c0e9-4d9e-8101-e73655e83f51 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:00.729653 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8c8e4509-c0e9-4d9e-8101-e73655e83f51 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:00.730131 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8c8e4509-c0e9-4d9e-8101-e73655e83f51 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:00.730750 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8c8e4509-c0e9-4d9e-8101-e73655e83f51 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:00.793594 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-8c8e4509-c0e9-4d9e-8101-e73655e83f51 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c92f7b4d09d74b8d93d739c1938694e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f7d505c7b9e4ca08d6437e6e529a039', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:00.802527 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-8c8e4509-c0e9-4d9e-8101-e73655e83f51 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] 10.222.0.16 "GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1" status: 200 len: 1345 microversion: 2.1 time: 0.087627 Oct 30 00:58:00.803098 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 111/229] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:00 2022] GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1 => generated 1345 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:01.827966 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-16632e9c-be46-4b28-8242-7e175dbfd39c tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:01.828713 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-16632e9c-be46-4b28-8242-7e175dbfd39c tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:01.836675 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-16632e9c-be46-4b28-8242-7e175dbfd39c tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:01.837270 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-16632e9c-be46-4b28-8242-7e175dbfd39c tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:01.837808 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-16632e9c-be46-4b28-8242-7e175dbfd39c tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:01.838650 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-16632e9c-be46-4b28-8242-7e175dbfd39c tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:01.839208 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-16632e9c-be46-4b28-8242-7e175dbfd39c tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:01.839703 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-16632e9c-be46-4b28-8242-7e175dbfd39c tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:01.907144 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-16632e9c-be46-4b28-8242-7e175dbfd39c tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c92f7b4d09d74b8d93d739c1938694e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f7d505c7b9e4ca08d6437e6e529a039', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:01.915893 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-16632e9c-be46-4b28-8242-7e175dbfd39c tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] 10.222.0.16 "GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1" status: 200 len: 1345 microversion: 2.1 time: 0.093709 Oct 30 00:58:01.916467 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 119/230] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:01 2022] GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1 => generated 1345 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:02.943570 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-039c1036-ea18-41b3-bc91-af912f6cd74c tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:02.944893 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-039c1036-ea18-41b3-bc91-af912f6cd74c tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:02.953347 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-039c1036-ea18-41b3-bc91-af912f6cd74c tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:02.953823 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-039c1036-ea18-41b3-bc91-af912f6cd74c tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:02.954246 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-039c1036-ea18-41b3-bc91-af912f6cd74c tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:02.954925 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-039c1036-ea18-41b3-bc91-af912f6cd74c tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:02.955366 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-039c1036-ea18-41b3-bc91-af912f6cd74c tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:02.955814 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-039c1036-ea18-41b3-bc91-af912f6cd74c tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:03.025169 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-039c1036-ea18-41b3-bc91-af912f6cd74c tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c92f7b4d09d74b8d93d739c1938694e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f7d505c7b9e4ca08d6437e6e529a039', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:03.033808 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-039c1036-ea18-41b3-bc91-af912f6cd74c tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] 10.222.0.16 "GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1" status: 200 len: 1345 microversion: 2.1 time: 0.097272 Oct 30 00:58:03.034371 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 112/231] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:02 2022] GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1 => generated 1345 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:04.060771 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-edb4b307-fc55-43c2-8c0c-cc962d67ab72 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:04.061369 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-edb4b307-fc55-43c2-8c0c-cc962d67ab72 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:04.067689 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-edb4b307-fc55-43c2-8c0c-cc962d67ab72 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:04.068211 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-edb4b307-fc55-43c2-8c0c-cc962d67ab72 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:04.068643 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-edb4b307-fc55-43c2-8c0c-cc962d67ab72 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:04.069332 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-edb4b307-fc55-43c2-8c0c-cc962d67ab72 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:04.069776 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-edb4b307-fc55-43c2-8c0c-cc962d67ab72 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:04.070193 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-edb4b307-fc55-43c2-8c0c-cc962d67ab72 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:04.131867 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-edb4b307-fc55-43c2-8c0c-cc962d67ab72 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c92f7b4d09d74b8d93d739c1938694e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f7d505c7b9e4ca08d6437e6e529a039', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:04.140716 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-edb4b307-fc55-43c2-8c0c-cc962d67ab72 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] 10.222.0.16 "GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1" status: 200 len: 1345 microversion: 2.1 time: 0.086221 Oct 30 00:58:04.141289 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 120/232] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:04 2022] GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1 => generated 1345 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:05.170111 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-ceddbce4-9569-4b5b-9c1f-8006e9ce386f tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:05.170638 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-ceddbce4-9569-4b5b-9c1f-8006e9ce386f tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:05.177495 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ceddbce4-9569-4b5b-9c1f-8006e9ce386f tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:05.177983 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ceddbce4-9569-4b5b-9c1f-8006e9ce386f tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:05.178414 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ceddbce4-9569-4b5b-9c1f-8006e9ce386f tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:05.179101 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ceddbce4-9569-4b5b-9c1f-8006e9ce386f tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:05.179573 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ceddbce4-9569-4b5b-9c1f-8006e9ce386f tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:05.180000 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ceddbce4-9569-4b5b-9c1f-8006e9ce386f tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:05.244419 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-ceddbce4-9569-4b5b-9c1f-8006e9ce386f tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c92f7b4d09d74b8d93d739c1938694e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f7d505c7b9e4ca08d6437e6e529a039', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:05.253522 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-ceddbce4-9569-4b5b-9c1f-8006e9ce386f tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] 10.222.0.16 "GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1" status: 200 len: 1345 microversion: 2.1 time: 0.088438 Oct 30 00:58:05.254087 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 113/233] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:05 2022] GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1 => generated 1345 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:06.280169 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-ebae1435-4162-4965-b9dd-b33ac0d51f3d tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:06.280806 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-ebae1435-4162-4965-b9dd-b33ac0d51f3d tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:06.287283 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ebae1435-4162-4965-b9dd-b33ac0d51f3d tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:06.287774 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ebae1435-4162-4965-b9dd-b33ac0d51f3d tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:06.288220 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ebae1435-4162-4965-b9dd-b33ac0d51f3d tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:06.288941 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ebae1435-4162-4965-b9dd-b33ac0d51f3d tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:06.289393 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ebae1435-4162-4965-b9dd-b33ac0d51f3d tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:06.289834 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ebae1435-4162-4965-b9dd-b33ac0d51f3d tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:06.362901 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-ebae1435-4162-4965-b9dd-b33ac0d51f3d tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c92f7b4d09d74b8d93d739c1938694e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f7d505c7b9e4ca08d6437e6e529a039', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:06.372986 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-ebae1435-4162-4965-b9dd-b33ac0d51f3d tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] 10.222.0.16 "GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1" status: 200 len: 1345 microversion: 2.1 time: 0.097508 Oct 30 00:58:06.384560 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 121/234] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:06 2022] GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1 => generated 1345 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:07.397703 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-6d74171d-0a90-4193-91c0-70d351167c9b tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:07.398382 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-6d74171d-0a90-4193-91c0-70d351167c9b tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:07.405093 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6d74171d-0a90-4193-91c0-70d351167c9b tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:07.405610 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6d74171d-0a90-4193-91c0-70d351167c9b tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:07.406151 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6d74171d-0a90-4193-91c0-70d351167c9b tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:07.406837 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6d74171d-0a90-4193-91c0-70d351167c9b tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:07.407276 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6d74171d-0a90-4193-91c0-70d351167c9b tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:07.407688 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6d74171d-0a90-4193-91c0-70d351167c9b tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:07.423325 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.wsgi [None req-6d74171d-0a90-4193-91c0-70d351167c9b tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] HTTP exception thrown: Instance 0f858c8d-6414-403e-b8d3-3ecd728e84a1 could not be found. Oct 30 00:58:07.428126 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-6d74171d-0a90-4193-91c0-70d351167c9b tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Request method failure captured: Oct 30 00:58:07.428126 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: request: GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1 HTTP/1.1 Oct 30 00:58:07.428126 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept: application/json Oct 30 00:58:07.428126 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept-Encoding: identity Oct 30 00:58:07.428126 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Connection: close Oct 30 00:58:07.428126 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Length: 0 Oct 30 00:58:07.428126 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Type: application/json Oct 30 00:58:07.428126 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Host: 10.222.0.16 Oct 30 00:58:07.428126 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Openstack-System-Scope: None Oct 30 00:58:07.428126 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: User-Agent: python-urllib3/1.26.12 Oct 30 00:58:07.428126 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Auth-Token: gAAAAABjXcufnJMThVIFRyo_dqgPV0Xh9Xs1vJ7zROHBxFqPi8yC60mikb2j5al3gL7z-tP4lpTr3bockIDyg88glmAd9P-1zvgHTtGSlslspLjdy00lSVtfdzN9dn3xBhJ7LuoTG5BGuPojJMSxIUPMs5eTAti-AS6OBVSCDMMXkddHqa4TJhk Oct 30 00:58:07.428126 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Id: None Oct 30 00:58:07.428126 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Name: None Oct 30 00:58:07.428126 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Identity-Status: Confirmed Oct 30 00:58:07.428126 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Is-Admin-Project: True Oct 30 00:58:07.428126 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Id: default Oct 30 00:58:07.428126 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Name: Default Oct 30 00:58:07.428126 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Id: 0f7d505c7b9e4ca08d6437e6e529a039 Oct 30 00:58:07.428126 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Name: tempest-DeleteServersTestJSON-531316675 Oct 30 00:58:07.428126 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Role: reader,member Oct 30 00:58:07.428126 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Roles: reader,member Oct 30 00:58:07.429350 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_0f7d505c7b9e4ca08d6437e6e529a039"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/0f7d505c7b9e4ca08d6437e6e529a039"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/0f7d505c7b9e4ca08d6437e6e529a039"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/0f7d505c7b9e4ca08d6437e6e529a039"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 00:58:07.429350 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant: tempest-DeleteServersTestJSON-531316675 Oct 30 00:58:07.429350 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Id: 0f7d505c7b9e4ca08d6437e6e529a039 Oct 30 00:58:07.429350 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Name: tempest-DeleteServersTestJSON-531316675 Oct 30 00:58:07.429350 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User: tempest-DeleteServersTestJSON-531316675-project Oct 30 00:58:07.429350 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Id: default Oct 30 00:58:07.429350 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Name: Default Oct 30 00:58:07.429350 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Id: c92f7b4d09d74b8d93d739c1938694e6 Oct 30 00:58:07.429350 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Name: tempest-DeleteServersTestJSON-531316675-project Oct 30 00:58:07.429350 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: method: > Oct 30 00:58:07.429350 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: exception: Instance 0f858c8d-6414-403e-b8d3-3ecd728e84a1 could not be found. Oct 30 00:58:07.429350 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: action_args: {'id': '0f858c8d-6414-403e-b8d3-3ecd728e84a1'} Oct 30 00:58:07.430059 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 00:58:07.430059 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 00:58:07.430059 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 30 00:58:07.430059 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 30 00:58:07.430059 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 30 00:58:07.430059 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 30 00:58:07.430059 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 30 00:58:07.430059 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 30 00:58:07.430059 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 30 00:58:07.430059 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 30 00:58:07.430059 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0f858c8d-6414-403e-b8d3-3ecd728e84a1 could not be found. Oct 30 00:58:07.430059 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 00:58:07.430059 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 00:58:07.430059 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 00:58:07.430059 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 00:58:07.430059 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 00:58:07.430059 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 00:58:07.430059 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 00:58:07.430059 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 00:58:07.430059 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 00:58:07.431505 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 00:58:07.431505 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 30 00:58:07.431505 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 30 00:58:07.431505 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 30 00:58:07.431505 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 30 00:58:07.431505 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 30 00:58:07.431505 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 30 00:58:07.431505 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0f858c8d-6414-403e-b8d3-3ecd728e84a1 could not be found. Oct 30 00:58:07.431505 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 00:58:07.431505 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 00:58:07.431505 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 00:58:07.431505 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 00:58:07.431505 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 00:58:07.431505 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 00:58:07.431505 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 00:58:07.431505 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 00:58:07.431505 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0f858c8d-6414-403e-b8d3-3ecd728e84a1 could not be found. Oct 30 00:58:07.431505 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 00:58:07.432648 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-6d74171d-0a90-4193-91c0-70d351167c9b tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Returning 404 to user: Instance 0f858c8d-6414-403e-b8d3-3ecd728e84a1 could not be found. {{(pid=118962) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 00:58:07.432648 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-6d74171d-0a90-4193-91c0-70d351167c9b tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] 10.222.0.16 "GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1" status: 404 len: 111 microversion: 2.1 time: 0.036576 Oct 30 00:58:07.432648 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 114/235] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:07 2022] GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1 => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 30 00:58:07.549785 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-907e9c6b-22d7-4d9a-b431-ac89adeae456 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:07.552111 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-907e9c6b-22d7-4d9a-b431-ac89adeae456 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:07.558651 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-907e9c6b-22d7-4d9a-b431-ac89adeae456 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:07.559477 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-907e9c6b-22d7-4d9a-b431-ac89adeae456 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:07.559987 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-907e9c6b-22d7-4d9a-b431-ac89adeae456 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:07.560950 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-907e9c6b-22d7-4d9a-b431-ac89adeae456 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:07.561426 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-907e9c6b-22d7-4d9a-b431-ac89adeae456 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:07.561965 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-907e9c6b-22d7-4d9a-b431-ac89adeae456 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:07.577917 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.wsgi [None req-907e9c6b-22d7-4d9a-b431-ac89adeae456 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] HTTP exception thrown: Instance 0f858c8d-6414-403e-b8d3-3ecd728e84a1 could not be found. Oct 30 00:58:07.588152 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-907e9c6b-22d7-4d9a-b431-ac89adeae456 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Request method failure captured: Oct 30 00:58:07.588152 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: request: GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1/os-volume_attachments HTTP/1.1 Oct 30 00:58:07.588152 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Accept: application/json Oct 30 00:58:07.588152 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Accept-Encoding: identity Oct 30 00:58:07.588152 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Connection: close Oct 30 00:58:07.588152 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Content-Length: 0 Oct 30 00:58:07.588152 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Content-Type: application/json Oct 30 00:58:07.588152 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Host: 10.222.0.16 Oct 30 00:58:07.588152 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Openstack-System-Scope: None Oct 30 00:58:07.588152 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: User-Agent: python-urllib3/1.26.12 Oct 30 00:58:07.588152 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Auth-Token: gAAAAABjXcufnJMThVIFRyo_dqgPV0Xh9Xs1vJ7zROHBxFqPi8yC60mikb2j5al3gL7z-tP4lpTr3bockIDyg88glmAd9P-1zvgHTtGSlslspLjdy00lSVtfdzN9dn3xBhJ7LuoTG5BGuPojJMSxIUPMs5eTAti-AS6OBVSCDMMXkddHqa4TJhk Oct 30 00:58:07.588152 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Domain-Id: None Oct 30 00:58:07.588152 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Domain-Name: None Oct 30 00:58:07.588152 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Identity-Status: Confirmed Oct 30 00:58:07.588152 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Is-Admin-Project: True Oct 30 00:58:07.588152 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Domain-Id: default Oct 30 00:58:07.588152 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Domain-Name: Default Oct 30 00:58:07.588152 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Id: 0f7d505c7b9e4ca08d6437e6e529a039 Oct 30 00:58:07.588152 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Name: tempest-DeleteServersTestJSON-531316675 Oct 30 00:58:07.588152 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Role: reader,member Oct 30 00:58:07.588152 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Roles: reader,member Oct 30 00:58:07.589584 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_0f7d505c7b9e4ca08d6437e6e529a039"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/0f7d505c7b9e4ca08d6437e6e529a039"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/0f7d505c7b9e4ca08d6437e6e529a039"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/0f7d505c7b9e4ca08d6437e6e529a039"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 00:58:07.589584 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant: tempest-DeleteServersTestJSON-531316675 Oct 30 00:58:07.589584 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant-Id: 0f7d505c7b9e4ca08d6437e6e529a039 Oct 30 00:58:07.589584 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant-Name: tempest-DeleteServersTestJSON-531316675 Oct 30 00:58:07.589584 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User: tempest-DeleteServersTestJSON-531316675-project Oct 30 00:58:07.589584 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Domain-Id: default Oct 30 00:58:07.589584 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Domain-Name: Default Oct 30 00:58:07.589584 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Id: c92f7b4d09d74b8d93d739c1938694e6 Oct 30 00:58:07.589584 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Name: tempest-DeleteServersTestJSON-531316675-project Oct 30 00:58:07.589584 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: method: > Oct 30 00:58:07.589584 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: exception: Instance 0f858c8d-6414-403e-b8d3-3ecd728e84a1 could not be found. Oct 30 00:58:07.589584 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: action_args: {'server_id': '0f858c8d-6414-403e-b8d3-3ecd728e84a1'} Oct 30 00:58:07.590397 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 00:58:07.590397 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 00:58:07.590397 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 30 00:58:07.590397 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 30 00:58:07.590397 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 30 00:58:07.590397 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 30 00:58:07.590397 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 30 00:58:07.590397 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 30 00:58:07.590397 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 30 00:58:07.590397 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 30 00:58:07.590397 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0f858c8d-6414-403e-b8d3-3ecd728e84a1 could not be found. Oct 30 00:58:07.590397 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 00:58:07.590397 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 00:58:07.590397 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 00:58:07.590397 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 00:58:07.590397 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 00:58:07.590397 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 00:58:07.590397 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 00:58:07.590397 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 00:58:07.590397 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 00:58:07.591742 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 00:58:07.591742 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Oct 30 00:58:07.591742 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 30 00:58:07.591742 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Oct 30 00:58:07.591742 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 30 00:58:07.591742 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 303, in index Oct 30 00:58:07.591742 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, server_id) Oct 30 00:58:07.591742 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 30 00:58:07.591742 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 30 00:58:07.591742 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0f858c8d-6414-403e-b8d3-3ecd728e84a1 could not be found. Oct 30 00:58:07.591742 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 00:58:07.591742 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 00:58:07.591742 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 00:58:07.591742 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 00:58:07.591742 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 00:58:07.591742 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 00:58:07.591742 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 00:58:07.591742 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 00:58:07.591742 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0f858c8d-6414-403e-b8d3-3ecd728e84a1 could not be found. Oct 30 00:58:07.591742 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 00:58:07.593100 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-907e9c6b-22d7-4d9a-b431-ac89adeae456 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Returning 404 to user: Instance 0f858c8d-6414-403e-b8d3-3ecd728e84a1 could not be found. {{(pid=118961) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 00:58:07.593100 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-907e9c6b-22d7-4d9a-b431-ac89adeae456 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] 10.222.0.16 "GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1/os-volume_attachments" status: 404 len: 111 microversion: 2.1 time: 0.044038 Oct 30 00:58:07.593100 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 122/236] 10.222.0.16 () {58 vars in 1302 bytes} [Sun Oct 30 00:58:07 2022] GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1/os-volume_attachments => generated 111 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 30 00:58:08.813418 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-45e89086-ea78-4fe8-8ef1-adffe37fc0e4 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:08.814434 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-45e89086-ea78-4fe8-8ef1-adffe37fc0e4 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:08.824048 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-45e89086-ea78-4fe8-8ef1-adffe37fc0e4 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:08.824048 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-45e89086-ea78-4fe8-8ef1-adffe37fc0e4 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:08.824048 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-45e89086-ea78-4fe8-8ef1-adffe37fc0e4 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:08.824048 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-45e89086-ea78-4fe8-8ef1-adffe37fc0e4 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:08.824396 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-45e89086-ea78-4fe8-8ef1-adffe37fc0e4 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:08.824396 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-45e89086-ea78-4fe8-8ef1-adffe37fc0e4 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:08.838091 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.wsgi [None req-45e89086-ea78-4fe8-8ef1-adffe37fc0e4 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] HTTP exception thrown: Instance 0f858c8d-6414-403e-b8d3-3ecd728e84a1 could not be found. Oct 30 00:58:08.839653 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-45e89086-ea78-4fe8-8ef1-adffe37fc0e4 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Request method failure captured: Oct 30 00:58:08.839653 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: request: DELETE /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1 HTTP/1.1 Oct 30 00:58:08.839653 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept: application/json Oct 30 00:58:08.839653 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept-Encoding: identity Oct 30 00:58:08.839653 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Connection: close Oct 30 00:58:08.839653 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Length: 0 Oct 30 00:58:08.839653 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Type: application/json Oct 30 00:58:08.839653 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Host: 10.222.0.16 Oct 30 00:58:08.839653 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Openstack-System-Scope: None Oct 30 00:58:08.839653 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: User-Agent: python-urllib3/1.26.12 Oct 30 00:58:08.839653 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Auth-Token: gAAAAABjXcufnJMThVIFRyo_dqgPV0Xh9Xs1vJ7zROHBxFqPi8yC60mikb2j5al3gL7z-tP4lpTr3bockIDyg88glmAd9P-1zvgHTtGSlslspLjdy00lSVtfdzN9dn3xBhJ7LuoTG5BGuPojJMSxIUPMs5eTAti-AS6OBVSCDMMXkddHqa4TJhk Oct 30 00:58:08.839653 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Id: None Oct 30 00:58:08.839653 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Name: None Oct 30 00:58:08.839653 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Identity-Status: Confirmed Oct 30 00:58:08.839653 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Is-Admin-Project: True Oct 30 00:58:08.839653 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Id: default Oct 30 00:58:08.839653 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Name: Default Oct 30 00:58:08.839653 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Id: 0f7d505c7b9e4ca08d6437e6e529a039 Oct 30 00:58:08.839653 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Name: tempest-DeleteServersTestJSON-531316675 Oct 30 00:58:08.839653 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Role: reader,member Oct 30 00:58:08.839653 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Roles: reader,member Oct 30 00:58:08.841099 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_0f7d505c7b9e4ca08d6437e6e529a039"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/0f7d505c7b9e4ca08d6437e6e529a039"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/0f7d505c7b9e4ca08d6437e6e529a039"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/0f7d505c7b9e4ca08d6437e6e529a039"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 00:58:08.841099 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant: tempest-DeleteServersTestJSON-531316675 Oct 30 00:58:08.841099 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Id: 0f7d505c7b9e4ca08d6437e6e529a039 Oct 30 00:58:08.841099 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Name: tempest-DeleteServersTestJSON-531316675 Oct 30 00:58:08.841099 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User: tempest-DeleteServersTestJSON-531316675-project Oct 30 00:58:08.841099 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Id: default Oct 30 00:58:08.841099 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Name: Default Oct 30 00:58:08.841099 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Id: c92f7b4d09d74b8d93d739c1938694e6 Oct 30 00:58:08.841099 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Name: tempest-DeleteServersTestJSON-531316675-project Oct 30 00:58:08.841099 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: method: > Oct 30 00:58:08.841099 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: exception: Instance 0f858c8d-6414-403e-b8d3-3ecd728e84a1 could not be found. Oct 30 00:58:08.841099 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: action_args: {'id': '0f858c8d-6414-403e-b8d3-3ecd728e84a1'} Oct 30 00:58:08.842464 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 00:58:08.842464 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 00:58:08.842464 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 30 00:58:08.842464 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 30 00:58:08.842464 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 30 00:58:08.842464 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 30 00:58:08.842464 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 30 00:58:08.842464 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 30 00:58:08.842464 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 30 00:58:08.842464 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 30 00:58:08.842464 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0f858c8d-6414-403e-b8d3-3ecd728e84a1 could not be found. Oct 30 00:58:08.842464 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 00:58:08.842464 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 00:58:08.842464 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 00:58:08.842464 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 00:58:08.842464 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 00:58:08.842464 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 00:58:08.842464 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 00:58:08.842464 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 00:58:08.842464 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 00:58:08.843592 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 00:58:08.843592 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 30 00:58:08.843592 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 30 00:58:08.843592 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 30 00:58:08.843592 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 30 00:58:08.843592 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 30 00:58:08.843592 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 30 00:58:08.843592 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 30 00:58:08.843592 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 30 00:58:08.843592 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0f858c8d-6414-403e-b8d3-3ecd728e84a1 could not be found. Oct 30 00:58:08.843592 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 00:58:08.843592 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 00:58:08.843592 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 00:58:08.843592 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 00:58:08.843592 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 00:58:08.843592 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 00:58:08.843592 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 00:58:08.843592 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 00:58:08.843592 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0f858c8d-6414-403e-b8d3-3ecd728e84a1 could not be found. Oct 30 00:58:08.843592 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 00:58:08.844974 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-45e89086-ea78-4fe8-8ef1-adffe37fc0e4 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Returning 404 to user: Instance 0f858c8d-6414-403e-b8d3-3ecd728e84a1 could not be found. {{(pid=118962) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 00:58:08.844974 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-45e89086-ea78-4fe8-8ef1-adffe37fc0e4 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] 10.222.0.16 "DELETE /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1" status: 404 len: 111 microversion: 2.1 time: 0.031668 Oct 30 00:58:08.844974 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 115/237] 10.222.0.16 () {58 vars in 1239 bytes} [Sun Oct 30 00:58:08 2022] DELETE /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 30 00:58:08.849908 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-3da9bf10-4f33-4bbe-8d61-ee9eb4572f29 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:08.850530 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-3da9bf10-4f33-4bbe-8d61-ee9eb4572f29 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] [instance: 0f858c8d-6414-403e-b8d3-3ecd728e84a1] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:08.856527 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3da9bf10-4f33-4bbe-8d61-ee9eb4572f29 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:08.857002 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3da9bf10-4f33-4bbe-8d61-ee9eb4572f29 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:08.857426 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3da9bf10-4f33-4bbe-8d61-ee9eb4572f29 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:08.858110 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3da9bf10-4f33-4bbe-8d61-ee9eb4572f29 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:08.858556 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3da9bf10-4f33-4bbe-8d61-ee9eb4572f29 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:08.859144 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3da9bf10-4f33-4bbe-8d61-ee9eb4572f29 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:08.872718 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.wsgi [None req-3da9bf10-4f33-4bbe-8d61-ee9eb4572f29 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] HTTP exception thrown: Instance 0f858c8d-6414-403e-b8d3-3ecd728e84a1 could not be found. Oct 30 00:58:08.874989 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-3da9bf10-4f33-4bbe-8d61-ee9eb4572f29 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Request method failure captured: Oct 30 00:58:08.874989 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: request: GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1 HTTP/1.1 Oct 30 00:58:08.874989 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Accept: application/json Oct 30 00:58:08.874989 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Accept-Encoding: identity Oct 30 00:58:08.874989 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Connection: close Oct 30 00:58:08.874989 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Content-Length: 0 Oct 30 00:58:08.874989 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Content-Type: application/json Oct 30 00:58:08.874989 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Host: 10.222.0.16 Oct 30 00:58:08.874989 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Openstack-System-Scope: None Oct 30 00:58:08.874989 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: User-Agent: python-urllib3/1.26.12 Oct 30 00:58:08.874989 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Auth-Token: gAAAAABjXcufnJMThVIFRyo_dqgPV0Xh9Xs1vJ7zROHBxFqPi8yC60mikb2j5al3gL7z-tP4lpTr3bockIDyg88glmAd9P-1zvgHTtGSlslspLjdy00lSVtfdzN9dn3xBhJ7LuoTG5BGuPojJMSxIUPMs5eTAti-AS6OBVSCDMMXkddHqa4TJhk Oct 30 00:58:08.874989 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Domain-Id: None Oct 30 00:58:08.874989 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Domain-Name: None Oct 30 00:58:08.874989 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Identity-Status: Confirmed Oct 30 00:58:08.874989 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Is-Admin-Project: True Oct 30 00:58:08.874989 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Domain-Id: default Oct 30 00:58:08.874989 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Domain-Name: Default Oct 30 00:58:08.874989 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Id: 0f7d505c7b9e4ca08d6437e6e529a039 Oct 30 00:58:08.874989 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Name: tempest-DeleteServersTestJSON-531316675 Oct 30 00:58:08.874989 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Role: reader,member Oct 30 00:58:08.874989 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Roles: reader,member Oct 30 00:58:08.876172 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_0f7d505c7b9e4ca08d6437e6e529a039"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/0f7d505c7b9e4ca08d6437e6e529a039"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/0f7d505c7b9e4ca08d6437e6e529a039"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/0f7d505c7b9e4ca08d6437e6e529a039"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 00:58:08.876172 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant: tempest-DeleteServersTestJSON-531316675 Oct 30 00:58:08.876172 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant-Id: 0f7d505c7b9e4ca08d6437e6e529a039 Oct 30 00:58:08.876172 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant-Name: tempest-DeleteServersTestJSON-531316675 Oct 30 00:58:08.876172 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User: tempest-DeleteServersTestJSON-531316675-project Oct 30 00:58:08.876172 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Domain-Id: default Oct 30 00:58:08.876172 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Domain-Name: Default Oct 30 00:58:08.876172 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Id: c92f7b4d09d74b8d93d739c1938694e6 Oct 30 00:58:08.876172 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Name: tempest-DeleteServersTestJSON-531316675-project Oct 30 00:58:08.876172 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: method: > Oct 30 00:58:08.876172 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: exception: Instance 0f858c8d-6414-403e-b8d3-3ecd728e84a1 could not be found. Oct 30 00:58:08.876172 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: action_args: {'id': '0f858c8d-6414-403e-b8d3-3ecd728e84a1'} Oct 30 00:58:08.876809 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 00:58:08.876809 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 00:58:08.876809 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 30 00:58:08.876809 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 30 00:58:08.876809 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 30 00:58:08.876809 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 30 00:58:08.876809 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 30 00:58:08.876809 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 30 00:58:08.876809 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 30 00:58:08.876809 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 30 00:58:08.876809 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0f858c8d-6414-403e-b8d3-3ecd728e84a1 could not be found. Oct 30 00:58:08.876809 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 00:58:08.876809 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 00:58:08.876809 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 00:58:08.876809 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 00:58:08.876809 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 00:58:08.876809 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 00:58:08.876809 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 00:58:08.876809 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 00:58:08.876809 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 00:58:08.878221 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 00:58:08.878221 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 30 00:58:08.878221 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 30 00:58:08.878221 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 30 00:58:08.878221 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 30 00:58:08.878221 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 30 00:58:08.878221 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 30 00:58:08.878221 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0f858c8d-6414-403e-b8d3-3ecd728e84a1 could not be found. Oct 30 00:58:08.878221 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 00:58:08.878221 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 00:58:08.878221 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 00:58:08.878221 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 00:58:08.878221 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 00:58:08.878221 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 00:58:08.878221 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 00:58:08.878221 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 00:58:08.878221 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0f858c8d-6414-403e-b8d3-3ecd728e84a1 could not be found. Oct 30 00:58:08.878221 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 00:58:08.879228 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-3da9bf10-4f33-4bbe-8d61-ee9eb4572f29 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] Returning 404 to user: Instance 0f858c8d-6414-403e-b8d3-3ecd728e84a1 could not be found. {{(pid=118961) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 00:58:08.879228 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-3da9bf10-4f33-4bbe-8d61-ee9eb4572f29 tempest-DeleteServersTestJSON-531316675 tempest-DeleteServersTestJSON-531316675-project] 10.222.0.16 "GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1" status: 404 len: 111 microversion: 2.1 time: 0.030382 Oct 30 00:58:08.879228 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 123/238] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:08 2022] GET /compute/v2.1/servers/0f858c8d-6414-403e-b8d3-3ecd728e84a1 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 30 00:58:21.186195 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_api:SELECT=20 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:58:21.187038 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell1:SELECT=48 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:58:21.327668 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-d8a798b0-43f9-4bc4-b4fc-1e65875e3be0 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-1857083194", "imageRef": "4c16f4f6-84cb-4463-b82c-b7c65f3826f0", "flavorRef": "452", "networks": [{"uuid": "58afba65-fd3f-46c8-9a69-fc8ae076237c"}]}} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:58:21.384964 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_api:SELECT=27,nova_api:UPDATE=1 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:58:21.387217 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell1:SELECT=66,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:58:21.404538 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-597b1b7f-cf33-41e5-aaf1-7f5c2820333e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1817063801", "imageRef": "4c16f4f6-84cb-4463-b82c-b7c65f3826f0", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:58:21.439528 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.network.neutron [None req-d8a798b0-43f9-4bc4-b4fc-1e65875e3be0 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] validate_networks() for [('58afba65-fd3f-46c8-9a69-fc8ae076237c', None, None, None, None, None)] {{(pid=118961) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 30 00:58:21.508296 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.network.neutron [None req-597b1b7f-cf33-41e5-aaf1-7f5c2820333e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] validate_networks() for [('cc0a9d8a-a1b9-41b6-86c4-62a6a6fe159a', None, None, None, None, None)] {{(pid=118962) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 30 00:58:22.187371 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-d8a798b0-43f9-4bc4-b4fc-1e65875e3be0 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Flavor limits 0:0:0 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 30 00:58:22.187826 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-d8a798b0-43f9-4bc4-b4fc-1e65875e3be0 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Image limits 0:0:0 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 30 00:58:22.188495 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-d8a798b0-43f9-4bc4-b4fc-1e65875e3be0 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Flavor pref 0:0:0 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 30 00:58:22.188944 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-d8a798b0-43f9-4bc4-b4fc-1e65875e3be0 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Image pref 0:0:0 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 30 00:58:22.189322 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-d8a798b0-43f9-4bc4-b4fc-1e65875e3be0 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 30 00:58:22.206245 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d8a798b0-43f9-4bc4-b4fc-1e65875e3be0 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:22.206649 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d8a798b0-43f9-4bc4-b4fc-1e65875e3be0 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:22.207091 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d8a798b0-43f9-4bc4-b4fc-1e65875e3be0 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:22.211906 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d8a798b0-43f9-4bc4-b4fc-1e65875e3be0 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:22.212457 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d8a798b0-43f9-4bc4-b4fc-1e65875e3be0 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:22.214089 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d8a798b0-43f9-4bc4-b4fc-1e65875e3be0 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:22.252621 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.quota [None req-d8a798b0-43f9-4bc4-b4fc-1e65875e3be0 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Getting quotas for project 2fbc73a853584aaba6109090bd6b1281. Resources: {'ram', 'instances', 'cores'} {{(pid=118961) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 30 00:58:22.258037 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.quota [None req-d8a798b0-43f9-4bc4-b4fc-1e65875e3be0 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Getting quotas for user ddeaa7a90ee7483590e6bc0d189e1e2a and project 2fbc73a853584aaba6109090bd6b1281. Resources: {'ram', 'instances', 'cores'} {{(pid=118961) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 30 00:58:22.274178 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-d8a798b0-43f9-4bc4-b4fc-1e65875e3be0 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Going to run 1 instances... {{(pid=118961) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 30 00:58:22.276557 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-d8a798b0-43f9-4bc4-b4fc-1e65875e3be0 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: 5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4c16f4f6-84cb-4463-b82c-b7c65f3826f0',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118961) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 30 00:58:22.301509 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.virt.hardware [None req-597b1b7f-cf33-41e5-aaf1-7f5c2820333e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Flavor limits 0:0:0 {{(pid=118962) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 30 00:58:22.302008 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.virt.hardware [None req-597b1b7f-cf33-41e5-aaf1-7f5c2820333e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Image limits 0:0:0 {{(pid=118962) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 30 00:58:22.302426 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.virt.hardware [None req-597b1b7f-cf33-41e5-aaf1-7f5c2820333e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Flavor pref 0:0:0 {{(pid=118962) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 30 00:58:22.302936 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.virt.hardware [None req-597b1b7f-cf33-41e5-aaf1-7f5c2820333e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Image pref 0:0:0 {{(pid=118962) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 30 00:58:22.303539 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.virt.hardware [None req-597b1b7f-cf33-41e5-aaf1-7f5c2820333e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118962) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 30 00:58:22.320982 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-597b1b7f-cf33-41e5-aaf1-7f5c2820333e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:22.323994 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-597b1b7f-cf33-41e5-aaf1-7f5c2820333e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:22.325692 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-597b1b7f-cf33-41e5-aaf1-7f5c2820333e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:22.327549 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-597b1b7f-cf33-41e5-aaf1-7f5c2820333e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:22.328226 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-597b1b7f-cf33-41e5-aaf1-7f5c2820333e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:22.328817 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-597b1b7f-cf33-41e5-aaf1-7f5c2820333e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:22.351660 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-d8a798b0-43f9-4bc4-b4fc-1e65875e3be0 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.030250 Oct 30 00:58:22.352083 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 124/239] 10.222.0.16 () {60 vars in 1147 bytes} [Sun Oct 30 00:58:21 2022] POST /compute/v2.1/servers => generated 380 bytes in 1032 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 30 00:58:22.366490 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-d1f9e211-2c75-47d9-99a2-840ae4466684 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:22.367445 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-d1f9e211-2c75-47d9-99a2-840ae4466684 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: 5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:22.370340 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.quota [None req-597b1b7f-cf33-41e5-aaf1-7f5c2820333e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Getting quotas for project f48276ac51dd4bb1912fc670433cf6c4. Resources: {'ram', 'instances', 'cores'} {{(pid=118962) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 30 00:58:22.380761 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.quota [None req-597b1b7f-cf33-41e5-aaf1-7f5c2820333e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Getting quotas for user 31b7342ecf07408cab6bb040f361fa1f and project f48276ac51dd4bb1912fc670433cf6c4. Resources: {'ram', 'instances', 'cores'} {{(pid=118962) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 30 00:58:22.410674 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-597b1b7f-cf33-41e5-aaf1-7f5c2820333e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Going to run 1 instances... {{(pid=118962) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 30 00:58:22.413140 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-597b1b7f-cf33-41e5-aaf1-7f5c2820333e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: c7be44ac-3b42-4ace-ac56-65c117e839c6] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4c16f4f6-84cb-4463-b82c-b7c65f3826f0',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118962) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 30 00:58:22.427277 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-d1f9e211-2c75-47d9-99a2-840ae4466684 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:22.435949 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-d1f9e211-2c75-47d9-99a2-840ae4466684 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a" status: 200 len: 1219 microversion: 2.1 time: 0.074820 Oct 30 00:58:22.437482 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 125/240] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:22 2022] GET /compute/v2.1/servers/5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a => generated 1219 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:22.519806 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-597b1b7f-cf33-41e5-aaf1-7f5c2820333e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 1.341598 Oct 30 00:58:22.527120 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 116/241] 10.222.0.16 () {60 vars in 1147 bytes} [Sun Oct 30 00:58:21 2022] POST /compute/v2.1/servers => generated 383 bytes in 1350 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 30 00:58:22.532725 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-0a7947d1-1b28-41fc-ae8f-a4a71b808305 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:22.533281 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-0a7947d1-1b28-41fc-ae8f-a4a71b808305 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: c7be44ac-3b42-4ace-ac56-65c117e839c6] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:22.654439 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-0a7947d1-1b28-41fc-ae8f-a4a71b808305 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:22.666552 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-0a7947d1-1b28-41fc-ae8f-a4a71b808305 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6" status: 200 len: 1222 microversion: 2.1 time: 0.138032 Oct 30 00:58:22.668014 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 126/242] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:22 2022] GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6 => generated 1222 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 00:58:22.896062 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-5470b7a9-9c35-4822-9339-3e25e8b5e84d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-485608473", "imageRef": "4c16f4f6-84cb-4463-b82c-b7c65f3826f0", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:58:22.981446 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.network.neutron [None req-5470b7a9-9c35-4822-9339-3e25e8b5e84d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] validate_networks() for [('cc0a9d8a-a1b9-41b6-86c4-62a6a6fe159a', None, None, None, None, None)] {{(pid=118962) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 30 00:58:23.457579 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.virt.hardware [None req-5470b7a9-9c35-4822-9339-3e25e8b5e84d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Flavor limits 0:0:0 {{(pid=118962) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 30 00:58:23.458146 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.virt.hardware [None req-5470b7a9-9c35-4822-9339-3e25e8b5e84d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Image limits 0:0:0 {{(pid=118962) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 30 00:58:23.458455 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.virt.hardware [None req-5470b7a9-9c35-4822-9339-3e25e8b5e84d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Flavor pref 0:0:0 {{(pid=118962) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 30 00:58:23.458858 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.virt.hardware [None req-5470b7a9-9c35-4822-9339-3e25e8b5e84d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Image pref 0:0:0 {{(pid=118962) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 30 00:58:23.459263 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.virt.hardware [None req-5470b7a9-9c35-4822-9339-3e25e8b5e84d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118962) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 30 00:58:23.462142 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-e4f1f8e7-e6de-4903-9b08-fa31295a841c tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:23.462750 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-e4f1f8e7-e6de-4903-9b08-fa31295a841c tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: 5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:23.469593 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5470b7a9-9c35-4822-9339-3e25e8b5e84d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:23.470231 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e4f1f8e7-e6de-4903-9b08-fa31295a841c tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:23.470610 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5470b7a9-9c35-4822-9339-3e25e8b5e84d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:23.470746 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e4f1f8e7-e6de-4903-9b08-fa31295a841c tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:23.471131 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e4f1f8e7-e6de-4903-9b08-fa31295a841c tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:23.471677 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5470b7a9-9c35-4822-9339-3e25e8b5e84d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:23.471890 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e4f1f8e7-e6de-4903-9b08-fa31295a841c tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:23.472308 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e4f1f8e7-e6de-4903-9b08-fa31295a841c tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:23.472727 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e4f1f8e7-e6de-4903-9b08-fa31295a841c tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:23.473474 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5470b7a9-9c35-4822-9339-3e25e8b5e84d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:23.474136 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5470b7a9-9c35-4822-9339-3e25e8b5e84d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:23.474777 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5470b7a9-9c35-4822-9339-3e25e8b5e84d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:23.504313 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.quota [None req-5470b7a9-9c35-4822-9339-3e25e8b5e84d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Getting quotas for project f48276ac51dd4bb1912fc670433cf6c4. Resources: {'ram', 'instances', 'cores'} {{(pid=118962) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 30 00:58:23.509987 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.quota [None req-5470b7a9-9c35-4822-9339-3e25e8b5e84d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Getting quotas for user 31b7342ecf07408cab6bb040f361fa1f and project f48276ac51dd4bb1912fc670433cf6c4. Resources: {'ram', 'instances', 'cores'} {{(pid=118962) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 30 00:58:23.526702 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-5470b7a9-9c35-4822-9339-3e25e8b5e84d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Going to run 1 instances... {{(pid=118962) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 30 00:58:23.529763 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-5470b7a9-9c35-4822-9339-3e25e8b5e84d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4c16f4f6-84cb-4463-b82c-b7c65f3826f0',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118962) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 30 00:58:23.547548 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-e4f1f8e7-e6de-4903-9b08-fa31295a841c tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:23.557883 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-e4f1f8e7-e6de-4903-9b08-fa31295a841c tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a" status: 200 len: 1223 microversion: 2.1 time: 0.099879 Oct 30 00:58:23.559426 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 127/243] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:23 2022] GET /compute/v2.1/servers/5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a => generated 1223 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 00:58:23.602127 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-5470b7a9-9c35-4822-9339-3e25e8b5e84d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 0.711163 Oct 30 00:58:23.603268 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 117/244] 10.222.0.16 () {60 vars in 1147 bytes} [Sun Oct 30 00:58:22 2022] POST /compute/v2.1/servers => generated 383 bytes in 713 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 30 00:58:23.614001 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-9b007b5f-c6f3-4622-86bc-bd12bcb9069b tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:23.614965 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-9b007b5f-c6f3-4622-86bc-bd12bcb9069b tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:23.712360 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-9b007b5f-c6f3-4622-86bc-bd12bcb9069b tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:23.740330 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-9b007b5f-c6f3-4622-86bc-bd12bcb9069b tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1221 microversion: 2.1 time: 0.127011 Oct 30 00:58:23.742168 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 128/245] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:23 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1221 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:24.585335 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-0e6fdd57-54e5-4799-b2af-d7c6a993c05a tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:24.587065 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-0e6fdd57-54e5-4799-b2af-d7c6a993c05a tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: 5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:24.592651 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0e6fdd57-54e5-4799-b2af-d7c6a993c05a tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:24.593200 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0e6fdd57-54e5-4799-b2af-d7c6a993c05a tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:24.593824 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0e6fdd57-54e5-4799-b2af-d7c6a993c05a tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:24.594523 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0e6fdd57-54e5-4799-b2af-d7c6a993c05a tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:24.594966 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0e6fdd57-54e5-4799-b2af-d7c6a993c05a tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:24.595379 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0e6fdd57-54e5-4799-b2af-d7c6a993c05a tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:24.659976 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-0e6fdd57-54e5-4799-b2af-d7c6a993c05a tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:24.672331 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-0e6fdd57-54e5-4799-b2af-d7c6a993c05a tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a" status: 200 len: 1223 microversion: 2.1 time: 0.091343 Oct 30 00:58:24.672913 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 118/246] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:24 2022] GET /compute/v2.1/servers/5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a => generated 1223 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:24.768945 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-3806416c-a12a-4341-8632-9deaf11d0d69 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:24.770127 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-3806416c-a12a-4341-8632-9deaf11d0d69 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:24.778537 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3806416c-a12a-4341-8632-9deaf11d0d69 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:24.779115 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3806416c-a12a-4341-8632-9deaf11d0d69 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:24.779651 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3806416c-a12a-4341-8632-9deaf11d0d69 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:24.780774 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3806416c-a12a-4341-8632-9deaf11d0d69 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:24.781065 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3806416c-a12a-4341-8632-9deaf11d0d69 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:24.781491 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3806416c-a12a-4341-8632-9deaf11d0d69 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:24.845372 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-3806416c-a12a-4341-8632-9deaf11d0d69 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:24.853189 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-3806416c-a12a-4341-8632-9deaf11d0d69 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1225 microversion: 2.1 time: 0.088966 Oct 30 00:58:24.854022 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 129/247] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:24 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1225 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:25.695286 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-788f0d25-2e99-4093-ad38-97196ec82488 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:25.695947 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-788f0d25-2e99-4093-ad38-97196ec82488 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: 5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:25.706635 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-788f0d25-2e99-4093-ad38-97196ec82488 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:25.707147 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-788f0d25-2e99-4093-ad38-97196ec82488 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:25.707583 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-788f0d25-2e99-4093-ad38-97196ec82488 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:25.708335 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-788f0d25-2e99-4093-ad38-97196ec82488 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:25.708790 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-788f0d25-2e99-4093-ad38-97196ec82488 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:25.709211 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-788f0d25-2e99-4093-ad38-97196ec82488 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:25.810584 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-788f0d25-2e99-4093-ad38-97196ec82488 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:25.826510 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-788f0d25-2e99-4093-ad38-97196ec82488 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a" status: 200 len: 1271 microversion: 2.1 time: 0.134526 Oct 30 00:58:25.827238 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 119/248] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:25 2022] GET /compute/v2.1/servers/5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a => generated 1271 bytes in 137 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:25.877400 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-321f0133-48e0-4952-a8f5-9dec504175b0 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:25.878773 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-321f0133-48e0-4952-a8f5-9dec504175b0 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:25.886860 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-321f0133-48e0-4952-a8f5-9dec504175b0 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:25.887599 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-321f0133-48e0-4952-a8f5-9dec504175b0 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:25.888525 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-321f0133-48e0-4952-a8f5-9dec504175b0 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:25.889494 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-321f0133-48e0-4952-a8f5-9dec504175b0 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:25.890183 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-321f0133-48e0-4952-a8f5-9dec504175b0 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:25.891209 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-321f0133-48e0-4952-a8f5-9dec504175b0 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:26.009699 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-321f0133-48e0-4952-a8f5-9dec504175b0 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:26.022162 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-321f0133-48e0-4952-a8f5-9dec504175b0 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1217 microversion: 2.1 time: 0.149897 Oct 30 00:58:26.024535 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 130/249] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:25 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1217 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:26.859019 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-60487d83-91ae-4387-9675-43540cffe1bb tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:26.859501 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-60487d83-91ae-4387-9675-43540cffe1bb tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: 5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:26.869704 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-60487d83-91ae-4387-9675-43540cffe1bb tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:26.870365 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-60487d83-91ae-4387-9675-43540cffe1bb tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:26.870815 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-60487d83-91ae-4387-9675-43540cffe1bb tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:26.871490 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-60487d83-91ae-4387-9675-43540cffe1bb tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:26.871961 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-60487d83-91ae-4387-9675-43540cffe1bb tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:26.872437 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-60487d83-91ae-4387-9675-43540cffe1bb tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:27.006251 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-60487d83-91ae-4387-9675-43540cffe1bb tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:27.017890 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-60487d83-91ae-4387-9675-43540cffe1bb tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a" status: 200 len: 1277 microversion: 2.1 time: 0.165285 Oct 30 00:58:27.020044 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 120/250] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:26 2022] GET /compute/v2.1/servers/5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a => generated 1277 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:27.049357 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-2b4084c3-274f-42cb-89b5-67eb0ab05d61 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:27.050322 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-2b4084c3-274f-42cb-89b5-67eb0ab05d61 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:27.071846 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2b4084c3-274f-42cb-89b5-67eb0ab05d61 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:27.074094 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2b4084c3-274f-42cb-89b5-67eb0ab05d61 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:27.075583 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2b4084c3-274f-42cb-89b5-67eb0ab05d61 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:27.077334 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2b4084c3-274f-42cb-89b5-67eb0ab05d61 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:27.078458 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2b4084c3-274f-42cb-89b5-67eb0ab05d61 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:27.079446 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2b4084c3-274f-42cb-89b5-67eb0ab05d61 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:27.272045 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-2b4084c3-274f-42cb-89b5-67eb0ab05d61 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:27.287180 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-2b4084c3-274f-42cb-89b5-67eb0ab05d61 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1217 microversion: 2.1 time: 0.241531 Oct 30 00:58:27.288359 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 131/251] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:27 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1217 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:28.041465 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-76eebd69-e9e1-407c-8e04-b96514c95f10 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:28.042174 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-76eebd69-e9e1-407c-8e04-b96514c95f10 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: 5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:28.058162 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-76eebd69-e9e1-407c-8e04-b96514c95f10 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:28.059448 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-76eebd69-e9e1-407c-8e04-b96514c95f10 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:28.063573 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-76eebd69-e9e1-407c-8e04-b96514c95f10 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:28.066952 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-76eebd69-e9e1-407c-8e04-b96514c95f10 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:28.067466 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-76eebd69-e9e1-407c-8e04-b96514c95f10 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:28.068114 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-76eebd69-e9e1-407c-8e04-b96514c95f10 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:28.314259 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-83db72f6-c8dd-4fb0-b910-ccad86b2db70 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:28.314872 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-83db72f6-c8dd-4fb0-b910-ccad86b2db70 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:28.328419 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-83db72f6-c8dd-4fb0-b910-ccad86b2db70 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:28.329265 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-83db72f6-c8dd-4fb0-b910-ccad86b2db70 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:28.329901 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-83db72f6-c8dd-4fb0-b910-ccad86b2db70 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:28.330973 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-83db72f6-c8dd-4fb0-b910-ccad86b2db70 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:28.336027 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-83db72f6-c8dd-4fb0-b910-ccad86b2db70 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:28.336751 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-83db72f6-c8dd-4fb0-b910-ccad86b2db70 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:28.550228 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-83db72f6-c8dd-4fb0-b910-ccad86b2db70 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:28.559432 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-83db72f6-c8dd-4fb0-b910-ccad86b2db70 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1291 microversion: 2.1 time: 0.249115 Oct 30 00:58:28.565888 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 132/252] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:28 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1291 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 00:58:28.574780 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-76eebd69-e9e1-407c-8e04-b96514c95f10 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:28.606730 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-76eebd69-e9e1-407c-8e04-b96514c95f10 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a" status: 200 len: 1319 microversion: 2.1 time: 0.568715 Oct 30 00:58:28.610582 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 121/253] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:28 2022] GET /compute/v2.1/servers/5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a => generated 1319 bytes in 574 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:29.317186 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [req-00e5e443-be3d-4bfd-a58b-871dfe27a39a req-bc9e830f-bd5a-4798-948b-ce099727189f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a", "tag": "762c8731-46b3-44de-8232-dda35195aa9c"}]} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:58:29.329733 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-00e5e443-be3d-4bfd-a58b-871dfe27a39a req-bc9e830f-bd5a-4798-948b-ce099727189f service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:29.330280 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-00e5e443-be3d-4bfd-a58b-871dfe27a39a req-bc9e830f-bd5a-4798-948b-ce099727189f service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:29.330977 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-00e5e443-be3d-4bfd-a58b-871dfe27a39a req-bc9e830f-bd5a-4798-948b-ce099727189f service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:29.349418 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.compute.server_external_events [req-00e5e443-be3d-4bfd-a58b-871dfe27a39a req-bc9e830f-bd5a-4798-948b-ce099727189f service nova] Creating event network-changed:762c8731-46b3-44de-8232-dda35195aa9c for instance 5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a on hv-cinder-maste Oct 30 00:58:29.370638 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [req-00e5e443-be3d-4bfd-a58b-871dfe27a39a req-bc9e830f-bd5a-4798-948b-ce099727189f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.064706 Oct 30 00:58:29.371380 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 133/254] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 00:58:29 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:58:29.589222 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-206caeb2-25e6-4a63-9505-0d5598166ec3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:29.589802 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-206caeb2-25e6-4a63-9505-0d5598166ec3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:29.597675 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-206caeb2-25e6-4a63-9505-0d5598166ec3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:29.598548 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-206caeb2-25e6-4a63-9505-0d5598166ec3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:29.598980 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-206caeb2-25e6-4a63-9505-0d5598166ec3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:29.599844 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-206caeb2-25e6-4a63-9505-0d5598166ec3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:29.600381 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-206caeb2-25e6-4a63-9505-0d5598166ec3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:29.601377 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-206caeb2-25e6-4a63-9505-0d5598166ec3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:29.635109 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-b4f6aef7-fcf8-471b-9bb9-c9dbd9d93717 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:29.636921 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-b4f6aef7-fcf8-471b-9bb9-c9dbd9d93717 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: 5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:29.649506 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b4f6aef7-fcf8-471b-9bb9-c9dbd9d93717 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:29.650008 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b4f6aef7-fcf8-471b-9bb9-c9dbd9d93717 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:29.650483 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b4f6aef7-fcf8-471b-9bb9-c9dbd9d93717 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:29.651198 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b4f6aef7-fcf8-471b-9bb9-c9dbd9d93717 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:29.651648 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b4f6aef7-fcf8-471b-9bb9-c9dbd9d93717 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:29.652128 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b4f6aef7-fcf8-471b-9bb9-c9dbd9d93717 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:29.942004 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-206caeb2-25e6-4a63-9505-0d5598166ec3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:29.953837 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-206caeb2-25e6-4a63-9505-0d5598166ec3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1279 microversion: 2.1 time: 0.368018 Oct 30 00:58:29.963872 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 122/255] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:29 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1279 bytes in 373 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:29.982100 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-b4f6aef7-fcf8-471b-9bb9-c9dbd9d93717 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:29.995520 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-b4f6aef7-fcf8-471b-9bb9-c9dbd9d93717 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a" status: 200 len: 1319 microversion: 2.1 time: 0.364564 Oct 30 00:58:29.996554 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 134/256] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:29 2022] GET /compute/v2.1/servers/5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a => generated 1319 bytes in 366 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 00:58:30.376704 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [req-520843d1-bc54-4f4e-9311-b160133dba32 req-c9c5eb9d-d0bd-403d-99d7-a1b889464524 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c7be44ac-3b42-4ace-ac56-65c117e839c6", "tag": "2a5ef263-4503-4e15-8a9c-3e1f7cdcf464"}]} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:58:30.386853 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-520843d1-bc54-4f4e-9311-b160133dba32 req-c9c5eb9d-d0bd-403d-99d7-a1b889464524 service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:30.387943 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-520843d1-bc54-4f4e-9311-b160133dba32 req-c9c5eb9d-d0bd-403d-99d7-a1b889464524 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:30.388760 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-520843d1-bc54-4f4e-9311-b160133dba32 req-c9c5eb9d-d0bd-403d-99d7-a1b889464524 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:30.407672 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.compute.server_external_events [req-520843d1-bc54-4f4e-9311-b160133dba32 req-c9c5eb9d-d0bd-403d-99d7-a1b889464524 service nova] Creating event network-changed:2a5ef263-4503-4e15-8a9c-3e1f7cdcf464 for instance c7be44ac-3b42-4ace-ac56-65c117e839c6 on hv-cinder-maste Oct 30 00:58:30.413029 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [req-520843d1-bc54-4f4e-9311-b160133dba32 req-c9c5eb9d-d0bd-403d-99d7-a1b889464524 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.040378 Oct 30 00:58:30.413626 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 123/257] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 00:58:30 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:58:30.985396 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-3fbabd55-1e8a-4826-b298-ec1940b6cea7 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:30.985940 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-3fbabd55-1e8a-4826-b298-ec1940b6cea7 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:30.992743 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3fbabd55-1e8a-4826-b298-ec1940b6cea7 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:30.993256 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3fbabd55-1e8a-4826-b298-ec1940b6cea7 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:30.993710 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3fbabd55-1e8a-4826-b298-ec1940b6cea7 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:30.994440 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3fbabd55-1e8a-4826-b298-ec1940b6cea7 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:30.994911 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3fbabd55-1e8a-4826-b298-ec1940b6cea7 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:30.995352 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3fbabd55-1e8a-4826-b298-ec1940b6cea7 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:31.020742 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-216d25a7-0da2-4a1b-81f7-2996656ae868 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:31.021450 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-216d25a7-0da2-4a1b-81f7-2996656ae868 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: 5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:31.027873 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-216d25a7-0da2-4a1b-81f7-2996656ae868 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:31.028346 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-216d25a7-0da2-4a1b-81f7-2996656ae868 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:31.028767 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-216d25a7-0da2-4a1b-81f7-2996656ae868 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:31.029426 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-216d25a7-0da2-4a1b-81f7-2996656ae868 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:31.029902 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-216d25a7-0da2-4a1b-81f7-2996656ae868 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:31.030306 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-216d25a7-0da2-4a1b-81f7-2996656ae868 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:31.181534 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-3fbabd55-1e8a-4826-b298-ec1940b6cea7 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:31.189513 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-3fbabd55-1e8a-4826-b298-ec1940b6cea7 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1321 microversion: 2.1 time: 0.210253 Oct 30 00:58:31.190081 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 135/258] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:30 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1321 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:31.210489 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-216d25a7-0da2-4a1b-81f7-2996656ae868 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:31.220622 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-216d25a7-0da2-4a1b-81f7-2996656ae868 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a" status: 200 len: 1319 microversion: 2.1 time: 0.204493 Oct 30 00:58:31.221433 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 124/259] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:31 2022] GET /compute/v2.1/servers/5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a => generated 1319 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:32.215801 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-04664ae6-fcd1-4238-a146-ef146022dd88 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:32.216792 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-04664ae6-fcd1-4238-a146-ef146022dd88 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:32.225698 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-04664ae6-fcd1-4238-a146-ef146022dd88 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:32.226412 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-04664ae6-fcd1-4238-a146-ef146022dd88 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:32.227051 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-04664ae6-fcd1-4238-a146-ef146022dd88 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:32.228113 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-04664ae6-fcd1-4238-a146-ef146022dd88 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:32.228807 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-04664ae6-fcd1-4238-a146-ef146022dd88 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:32.229439 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-04664ae6-fcd1-4238-a146-ef146022dd88 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:32.235077 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell0:SELECT=2 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:58:32.245782 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-c8cfdb76-36be-4cb9-a579-7f90239b0d61 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:32.246566 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-c8cfdb76-36be-4cb9-a579-7f90239b0d61 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: 5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:32.252879 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c8cfdb76-36be-4cb9-a579-7f90239b0d61 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:32.253354 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c8cfdb76-36be-4cb9-a579-7f90239b0d61 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:32.253798 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c8cfdb76-36be-4cb9-a579-7f90239b0d61 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:32.254472 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c8cfdb76-36be-4cb9-a579-7f90239b0d61 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:32.254921 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c8cfdb76-36be-4cb9-a579-7f90239b0d61 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:32.255326 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c8cfdb76-36be-4cb9-a579-7f90239b0d61 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:32.425158 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-04664ae6-fcd1-4238-a146-ef146022dd88 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:32.432793 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-c8cfdb76-36be-4cb9-a579-7f90239b0d61 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:32.433065 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-04664ae6-fcd1-4238-a146-ef146022dd88 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1321 microversion: 2.1 time: 0.223014 Oct 30 00:58:32.433641 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 136/260] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:32 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1321 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:32.441112 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-c8cfdb76-36be-4cb9-a579-7f90239b0d61 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a" status: 200 len: 1319 microversion: 2.1 time: 0.199721 Oct 30 00:58:32.441681 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 125/261] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:32 2022] GET /compute/v2.1/servers/5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a => generated 1319 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:33.460653 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-98ac5a76-8fd1-4f04-bea3-b2c606c07012 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:33.461510 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-98ac5a76-8fd1-4f04-bea3-b2c606c07012 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:33.467493 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-15b6bc66-c73e-4638-b600-60acc619dfee tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:33.468487 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-15b6bc66-c73e-4638-b600-60acc619dfee tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: 5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:33.469795 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-98ac5a76-8fd1-4f04-bea3-b2c606c07012 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:33.470287 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-98ac5a76-8fd1-4f04-bea3-b2c606c07012 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:33.470726 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-98ac5a76-8fd1-4f04-bea3-b2c606c07012 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:33.471425 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-98ac5a76-8fd1-4f04-bea3-b2c606c07012 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:33.471921 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-98ac5a76-8fd1-4f04-bea3-b2c606c07012 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:33.472342 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-98ac5a76-8fd1-4f04-bea3-b2c606c07012 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:33.474675 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-15b6bc66-c73e-4638-b600-60acc619dfee tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:33.475153 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-15b6bc66-c73e-4638-b600-60acc619dfee tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:33.475579 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-15b6bc66-c73e-4638-b600-60acc619dfee tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:33.476273 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-15b6bc66-c73e-4638-b600-60acc619dfee tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:33.476735 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-15b6bc66-c73e-4638-b600-60acc619dfee tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:33.477144 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-15b6bc66-c73e-4638-b600-60acc619dfee tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:33.491754 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell0:SELECT=4 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:58:33.522497 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell0:SELECT=12 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:58:33.667657 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-98ac5a76-8fd1-4f04-bea3-b2c606c07012 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:33.672000 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-15b6bc66-c73e-4638-b600-60acc619dfee tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:33.676125 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-98ac5a76-8fd1-4f04-bea3-b2c606c07012 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1321 microversion: 2.1 time: 0.221943 Oct 30 00:58:33.677259 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 137/262] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:33 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1321 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:33.679842 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-15b6bc66-c73e-4638-b600-60acc619dfee tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a" status: 200 len: 1319 microversion: 2.1 time: 0.218963 Oct 30 00:58:33.680380 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 126/263] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:33 2022] GET /compute/v2.1/servers/5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a => generated 1319 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:34.703014 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-341efed2-d434-4a88-8450-e17d48ceded3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:34.704073 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-341efed2-d434-4a88-8450-e17d48ceded3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:34.707047 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell0:SELECT=12 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:58:34.707301 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-dc6ace38-1b7f-41da-bb1c-c69d44828c64 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:34.708541 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-dc6ace38-1b7f-41da-bb1c-c69d44828c64 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: 5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:34.716936 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-341efed2-d434-4a88-8450-e17d48ceded3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:34.717736 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-341efed2-d434-4a88-8450-e17d48ceded3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:34.718447 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-341efed2-d434-4a88-8450-e17d48ceded3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:34.719114 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dc6ace38-1b7f-41da-bb1c-c69d44828c64 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:34.719443 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-341efed2-d434-4a88-8450-e17d48ceded3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:34.719624 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dc6ace38-1b7f-41da-bb1c-c69d44828c64 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:34.719846 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-341efed2-d434-4a88-8450-e17d48ceded3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:34.720044 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dc6ace38-1b7f-41da-bb1c-c69d44828c64 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:34.720260 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-341efed2-d434-4a88-8450-e17d48ceded3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:34.720762 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dc6ace38-1b7f-41da-bb1c-c69d44828c64 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:34.721218 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dc6ace38-1b7f-41da-bb1c-c69d44828c64 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:34.721638 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dc6ace38-1b7f-41da-bb1c-c69d44828c64 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:34.912209 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-dc6ace38-1b7f-41da-bb1c-c69d44828c64 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:34.919166 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-341efed2-d434-4a88-8450-e17d48ceded3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:34.919984 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-dc6ace38-1b7f-41da-bb1c-c69d44828c64 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a" status: 200 len: 1319 microversion: 2.1 time: 0.219754 Oct 30 00:58:34.920568 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 127/264] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:34 2022] GET /compute/v2.1/servers/5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a => generated 1319 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:34.927002 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-341efed2-d434-4a88-8450-e17d48ceded3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1321 microversion: 2.1 time: 0.230835 Oct 30 00:58:34.927547 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 138/265] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:34 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1321 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:35.943758 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-c9e8a608-bdca-4169-9093-cdee27d25224 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:35.946274 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-c9e8a608-bdca-4169-9093-cdee27d25224 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: 5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:35.950794 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-157c18eb-484d-41e2-8515-e9570850239c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:35.952382 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-157c18eb-484d-41e2-8515-e9570850239c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:35.955480 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c9e8a608-bdca-4169-9093-cdee27d25224 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:35.956551 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c9e8a608-bdca-4169-9093-cdee27d25224 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:35.957057 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c9e8a608-bdca-4169-9093-cdee27d25224 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:35.958188 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c9e8a608-bdca-4169-9093-cdee27d25224 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:35.958818 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c9e8a608-bdca-4169-9093-cdee27d25224 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:35.959524 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c9e8a608-bdca-4169-9093-cdee27d25224 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:35.973629 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-157c18eb-484d-41e2-8515-e9570850239c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:35.974140 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-157c18eb-484d-41e2-8515-e9570850239c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:35.974586 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-157c18eb-484d-41e2-8515-e9570850239c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:35.975302 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-157c18eb-484d-41e2-8515-e9570850239c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:35.975955 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-157c18eb-484d-41e2-8515-e9570850239c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:35.976411 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-157c18eb-484d-41e2-8515-e9570850239c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:36.154381 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-c9e8a608-bdca-4169-9093-cdee27d25224 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:36.163430 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-c9e8a608-bdca-4169-9093-cdee27d25224 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a" status: 200 len: 1319 microversion: 2.1 time: 0.223724 Oct 30 00:58:36.169155 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-157c18eb-484d-41e2-8515-e9570850239c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:36.176123 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 128/266] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:35 2022] GET /compute/v2.1/servers/5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a => generated 1319 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:36.182632 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-157c18eb-484d-41e2-8515-e9570850239c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1321 microversion: 2.1 time: 0.236025 Oct 30 00:58:36.183209 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 139/267] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:35 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1321 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:37.190400 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-afd42c53-8c53-4f2c-83c5-ff022bf717b4 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:37.192970 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-afd42c53-8c53-4f2c-83c5-ff022bf717b4 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: 5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:37.205210 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-afd42c53-8c53-4f2c-83c5-ff022bf717b4 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:37.205737 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-251290e4-f25a-4a68-93d1-a5ae3bbf9300 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:37.206477 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-251290e4-f25a-4a68-93d1-a5ae3bbf9300 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:37.207804 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-afd42c53-8c53-4f2c-83c5-ff022bf717b4 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:37.209126 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-afd42c53-8c53-4f2c-83c5-ff022bf717b4 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:37.211428 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-afd42c53-8c53-4f2c-83c5-ff022bf717b4 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:37.213169 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-afd42c53-8c53-4f2c-83c5-ff022bf717b4 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:37.214143 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-afd42c53-8c53-4f2c-83c5-ff022bf717b4 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:37.216357 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-251290e4-f25a-4a68-93d1-a5ae3bbf9300 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:37.216851 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-251290e4-f25a-4a68-93d1-a5ae3bbf9300 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:37.217599 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-251290e4-f25a-4a68-93d1-a5ae3bbf9300 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:37.218471 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-251290e4-f25a-4a68-93d1-a5ae3bbf9300 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:37.219353 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-251290e4-f25a-4a68-93d1-a5ae3bbf9300 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:37.221636 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-251290e4-f25a-4a68-93d1-a5ae3bbf9300 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:37.472292 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-afd42c53-8c53-4f2c-83c5-ff022bf717b4 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:37.480337 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-afd42c53-8c53-4f2c-83c5-ff022bf717b4 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a" status: 200 len: 1319 microversion: 2.1 time: 0.297446 Oct 30 00:58:37.481033 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 129/268] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:37 2022] GET /compute/v2.1/servers/5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a => generated 1319 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:37.502994 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-251290e4-f25a-4a68-93d1-a5ae3bbf9300 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:37.510680 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-251290e4-f25a-4a68-93d1-a5ae3bbf9300 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1321 microversion: 2.1 time: 0.308722 Oct 30 00:58:37.524195 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 140/269] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:37 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1321 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:37.730421 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [req-5905504b-17e2-4354-9f26-986dbbf61045 req-cb3c1220-f06c-4e37-8561-db626e5738b7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a", "name": "network-vif-plugged", "status": "completed", "tag": "762c8731-46b3-44de-8232-dda35195aa9c"}]} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:58:37.748605 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-5905504b-17e2-4354-9f26-986dbbf61045 req-cb3c1220-f06c-4e37-8561-db626e5738b7 service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:37.749261 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-5905504b-17e2-4354-9f26-986dbbf61045 req-cb3c1220-f06c-4e37-8561-db626e5738b7 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:37.749903 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-5905504b-17e2-4354-9f26-986dbbf61045 req-cb3c1220-f06c-4e37-8561-db626e5738b7 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:37.764116 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.compute.server_external_events [req-5905504b-17e2-4354-9f26-986dbbf61045 req-cb3c1220-f06c-4e37-8561-db626e5738b7 service nova] Creating event network-vif-plugged:762c8731-46b3-44de-8232-dda35195aa9c for instance 5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a on hv-cinder-maste Oct 30 00:58:37.770054 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [req-5905504b-17e2-4354-9f26-986dbbf61045 req-cb3c1220-f06c-4e37-8561-db626e5738b7 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.044715 Oct 30 00:58:37.771092 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 130/270] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 00:58:37 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 30 00:58:38.507311 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-b1346d62-6602-4cd1-9bd1-0b579833636f tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:38.508514 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-b1346d62-6602-4cd1-9bd1-0b579833636f tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: 5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:38.516962 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b1346d62-6602-4cd1-9bd1-0b579833636f tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:38.517920 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b1346d62-6602-4cd1-9bd1-0b579833636f tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:38.518590 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b1346d62-6602-4cd1-9bd1-0b579833636f tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:38.520385 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b1346d62-6602-4cd1-9bd1-0b579833636f tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:38.521018 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b1346d62-6602-4cd1-9bd1-0b579833636f tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:38.521597 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b1346d62-6602-4cd1-9bd1-0b579833636f tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:38.534884 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-ebcda4ad-48c2-4d0c-96f0-46dd6f4ea2f5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:38.535641 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-ebcda4ad-48c2-4d0c-96f0-46dd6f4ea2f5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:38.547232 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ebcda4ad-48c2-4d0c-96f0-46dd6f4ea2f5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:38.547760 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ebcda4ad-48c2-4d0c-96f0-46dd6f4ea2f5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:38.548166 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ebcda4ad-48c2-4d0c-96f0-46dd6f4ea2f5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:38.550232 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ebcda4ad-48c2-4d0c-96f0-46dd6f4ea2f5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:38.552957 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ebcda4ad-48c2-4d0c-96f0-46dd6f4ea2f5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:38.554074 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ebcda4ad-48c2-4d0c-96f0-46dd6f4ea2f5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:38.829740 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-b1346d62-6602-4cd1-9bd1-0b579833636f tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:38.838359 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-b1346d62-6602-4cd1-9bd1-0b579833636f tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a" status: 200 len: 1319 microversion: 2.1 time: 0.338719 Oct 30 00:58:38.838939 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 141/271] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:38 2022] GET /compute/v2.1/servers/5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a => generated 1319 bytes in 341 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:38.850965 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [req-9c790b60-6ad7-451b-84e8-9851da9655b1 req-263def06-5d7e-4724-8698-579b87159c8e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "60b2f6e0-a160-4d3f-b036-38597b60da68", "tag": "88fd74d2-892d-4aaf-b602-415c64c5b9aa"}]} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:58:38.854862 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-ebcda4ad-48c2-4d0c-96f0-46dd6f4ea2f5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:38.869391 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-9c790b60-6ad7-451b-84e8-9851da9655b1 req-263def06-5d7e-4724-8698-579b87159c8e service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:38.869971 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-9c790b60-6ad7-451b-84e8-9851da9655b1 req-263def06-5d7e-4724-8698-579b87159c8e service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:38.870519 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-9c790b60-6ad7-451b-84e8-9851da9655b1 req-263def06-5d7e-4724-8698-579b87159c8e service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:38.875551 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-ebcda4ad-48c2-4d0c-96f0-46dd6f4ea2f5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1321 microversion: 2.1 time: 0.345444 Oct 30 00:58:38.877031 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 131/272] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:38 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1321 bytes in 348 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:38.913091 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.compute.server_external_events [req-9c790b60-6ad7-451b-84e8-9851da9655b1 req-263def06-5d7e-4724-8698-579b87159c8e service nova] Creating event network-changed:88fd74d2-892d-4aaf-b602-415c64c5b9aa for instance 60b2f6e0-a160-4d3f-b036-38597b60da68 on hv-cinder-maste Oct 30 00:58:38.923987 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [req-9c790b60-6ad7-451b-84e8-9851da9655b1 req-263def06-5d7e-4724-8698-579b87159c8e service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.077500 Oct 30 00:58:38.924589 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 142/273] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 00:58:38 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:58:39.865882 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-7ea41168-bf5f-40ae-a7d3-d353c7383bdf tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:39.867059 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-7ea41168-bf5f-40ae-a7d3-d353c7383bdf tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: 5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:39.874029 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7ea41168-bf5f-40ae-a7d3-d353c7383bdf tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:39.874746 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7ea41168-bf5f-40ae-a7d3-d353c7383bdf tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:39.875512 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7ea41168-bf5f-40ae-a7d3-d353c7383bdf tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:39.876550 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7ea41168-bf5f-40ae-a7d3-d353c7383bdf tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:39.877173 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7ea41168-bf5f-40ae-a7d3-d353c7383bdf tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:39.877759 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7ea41168-bf5f-40ae-a7d3-d353c7383bdf tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:39.902572 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-a34f7e3c-13fd-4ab9-94a6-af369180c5dc tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:39.903286 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-a34f7e3c-13fd-4ab9-94a6-af369180c5dc tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:39.913955 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a34f7e3c-13fd-4ab9-94a6-af369180c5dc tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:39.914549 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a34f7e3c-13fd-4ab9-94a6-af369180c5dc tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:39.915263 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a34f7e3c-13fd-4ab9-94a6-af369180c5dc tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:39.916689 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a34f7e3c-13fd-4ab9-94a6-af369180c5dc tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:39.917742 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a34f7e3c-13fd-4ab9-94a6-af369180c5dc tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:39.918462 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a34f7e3c-13fd-4ab9-94a6-af369180c5dc tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:40.188202 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-7ea41168-bf5f-40ae-a7d3-d353c7383bdf tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:40.197595 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-7ea41168-bf5f-40ae-a7d3-d353c7383bdf tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a" status: 200 len: 1319 microversion: 2.1 time: 0.335468 Oct 30 00:58:40.198620 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 132/274] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:39 2022] GET /compute/v2.1/servers/5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a => generated 1319 bytes in 337 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 00:58:40.201017 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-a34f7e3c-13fd-4ab9-94a6-af369180c5dc tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:40.209757 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-a34f7e3c-13fd-4ab9-94a6-af369180c5dc tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1321 microversion: 2.1 time: 0.312726 Oct 30 00:58:40.210339 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 143/275] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:39 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1321 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 00:58:41.224490 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-3563e9ba-590c-4504-836f-37f5262f06d8 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:41.224490 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-3563e9ba-590c-4504-836f-37f5262f06d8 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: 5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:41.233804 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3563e9ba-590c-4504-836f-37f5262f06d8 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:41.234649 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3563e9ba-590c-4504-836f-37f5262f06d8 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:41.235230 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3563e9ba-590c-4504-836f-37f5262f06d8 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:41.236113 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3563e9ba-590c-4504-836f-37f5262f06d8 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:41.236805 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3563e9ba-590c-4504-836f-37f5262f06d8 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:41.237317 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3563e9ba-590c-4504-836f-37f5262f06d8 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:41.245350 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-1f27998e-fb46-47bb-a011-c73db01efc4b tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:41.248966 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-1f27998e-fb46-47bb-a011-c73db01efc4b tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:41.266668 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1f27998e-fb46-47bb-a011-c73db01efc4b tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:41.269358 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1f27998e-fb46-47bb-a011-c73db01efc4b tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:41.270023 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1f27998e-fb46-47bb-a011-c73db01efc4b tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:41.271573 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1f27998e-fb46-47bb-a011-c73db01efc4b tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:41.272208 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1f27998e-fb46-47bb-a011-c73db01efc4b tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:41.272949 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1f27998e-fb46-47bb-a011-c73db01efc4b tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:41.468352 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-3563e9ba-590c-4504-836f-37f5262f06d8 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:41.481467 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-3563e9ba-590c-4504-836f-37f5262f06d8 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a" status: 200 len: 1508 microversion: 2.1 time: 0.263132 Oct 30 00:58:41.494176 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-1f27998e-fb46-47bb-a011-c73db01efc4b tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:41.502760 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 133/276] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:41 2022] GET /compute/v2.1/servers/5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a => generated 1508 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:41.503754 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-1f27998e-fb46-47bb-a011-c73db01efc4b tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1321 microversion: 2.1 time: 0.273692 Oct 30 00:58:41.504743 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 144/277] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:41 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1321 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:42.541668 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-a8f88b17-5722-4811-a546-20d063e65e50 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:42.543036 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-a8f88b17-5722-4811-a546-20d063e65e50 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:42.550633 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a8f88b17-5722-4811-a546-20d063e65e50 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:42.551329 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a8f88b17-5722-4811-a546-20d063e65e50 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:42.552001 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a8f88b17-5722-4811-a546-20d063e65e50 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:42.553230 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a8f88b17-5722-4811-a546-20d063e65e50 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:42.554098 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a8f88b17-5722-4811-a546-20d063e65e50 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:42.554680 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a8f88b17-5722-4811-a546-20d063e65e50 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:42.762658 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-a8f88b17-5722-4811-a546-20d063e65e50 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:42.770976 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-a8f88b17-5722-4811-a546-20d063e65e50 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1321 microversion: 2.1 time: 0.234116 Oct 30 00:58:42.771796 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 134/278] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:42 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1321 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:43.798449 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-5478059e-9898-4ac0-92c6-bdef684982cb tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:43.799518 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-5478059e-9898-4ac0-92c6-bdef684982cb tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:43.807752 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5478059e-9898-4ac0-92c6-bdef684982cb tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:43.808717 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5478059e-9898-4ac0-92c6-bdef684982cb tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:43.809214 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5478059e-9898-4ac0-92c6-bdef684982cb tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:43.810197 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5478059e-9898-4ac0-92c6-bdef684982cb tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:43.810720 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5478059e-9898-4ac0-92c6-bdef684982cb tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:43.811257 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5478059e-9898-4ac0-92c6-bdef684982cb tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:43.832291 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell1:SELECT=100 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:58:43.833193 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-eeddb7e6-6e62-4fdd-bba5-23345fbd083d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-708535080", "imageRef": "4c16f4f6-84cb-4463-b82c-b7c65f3826f0", "flavorRef": "452", "networks": [{"uuid": "e79f970e-5030-4a3e-9305-668e891be9fd"}]}} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:58:43.889324 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.network.neutron [None req-eeddb7e6-6e62-4fdd-bba5-23345fbd083d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] validate_networks() for [('e79f970e-5030-4a3e-9305-668e891be9fd', None, None, None, None, None)] {{(pid=118962) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 30 00:58:44.263851 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-5478059e-9898-4ac0-92c6-bdef684982cb tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:44.271881 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-5478059e-9898-4ac0-92c6-bdef684982cb tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1321 microversion: 2.1 time: 0.479327 Oct 30 00:58:44.272465 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 145/279] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:43 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1321 bytes in 481 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:44.518697 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.virt.hardware [None req-eeddb7e6-6e62-4fdd-bba5-23345fbd083d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Flavor limits 0:0:0 {{(pid=118962) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 30 00:58:44.519118 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.virt.hardware [None req-eeddb7e6-6e62-4fdd-bba5-23345fbd083d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Image limits 0:0:0 {{(pid=118962) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 30 00:58:44.519580 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.virt.hardware [None req-eeddb7e6-6e62-4fdd-bba5-23345fbd083d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Flavor pref 0:0:0 {{(pid=118962) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 30 00:58:44.520007 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.virt.hardware [None req-eeddb7e6-6e62-4fdd-bba5-23345fbd083d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Image pref 0:0:0 {{(pid=118962) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 30 00:58:44.520411 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.virt.hardware [None req-eeddb7e6-6e62-4fdd-bba5-23345fbd083d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118962) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 30 00:58:44.529407 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-eeddb7e6-6e62-4fdd-bba5-23345fbd083d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:44.529951 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-eeddb7e6-6e62-4fdd-bba5-23345fbd083d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:44.530406 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-eeddb7e6-6e62-4fdd-bba5-23345fbd083d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:44.531132 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-eeddb7e6-6e62-4fdd-bba5-23345fbd083d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:44.531734 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-eeddb7e6-6e62-4fdd-bba5-23345fbd083d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:44.532098 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-eeddb7e6-6e62-4fdd-bba5-23345fbd083d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:44.539023 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell1:SELECT=100 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:58:44.550791 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.quota [None req-eeddb7e6-6e62-4fdd-bba5-23345fbd083d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Getting quotas for project dd16277cbc344c9aa8ffbc45ce1a16d2. Resources: {'ram', 'instances', 'cores'} {{(pid=118962) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 30 00:58:44.554794 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.quota [None req-eeddb7e6-6e62-4fdd-bba5-23345fbd083d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Getting quotas for user 5f198d5e999b487bad75a6b423fe1adc and project dd16277cbc344c9aa8ffbc45ce1a16d2. Resources: {'ram', 'instances', 'cores'} {{(pid=118962) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 30 00:58:44.566428 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-eeddb7e6-6e62-4fdd-bba5-23345fbd083d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Going to run 1 instances... {{(pid=118962) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 30 00:58:44.568902 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-eeddb7e6-6e62-4fdd-bba5-23345fbd083d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 6eaaf253-16e7-4c62-a434-a4b8f5d7b536] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4c16f4f6-84cb-4463-b82c-b7c65f3826f0',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118962) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 30 00:58:44.617815 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-eeddb7e6-6e62-4fdd-bba5-23345fbd083d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.788860 Oct 30 00:58:44.618467 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 135/280] 10.222.0.16 () {60 vars in 1147 bytes} [Sun Oct 30 00:58:43 2022] POST /compute/v2.1/servers => generated 380 bytes in 790 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 30 00:58:44.629861 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-91ec27d7-124a-4a3a-9e29-63274cb28107 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:44.630472 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-91ec27d7-124a-4a3a-9e29-63274cb28107 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 6eaaf253-16e7-4c62-a434-a4b8f5d7b536] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:44.686570 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-91ec27d7-124a-4a3a-9e29-63274cb28107 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:44.693274 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-91ec27d7-124a-4a3a-9e29-63274cb28107 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536" status: 200 len: 1213 microversion: 2.1 time: 0.067283 Oct 30 00:58:44.693928 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 146/281] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:44 2022] GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536 => generated 1213 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:45.294311 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-0fd58700-23eb-4b3c-9709-4c4ecc03ce35 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:45.295504 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-0fd58700-23eb-4b3c-9709-4c4ecc03ce35 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:45.303799 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0fd58700-23eb-4b3c-9709-4c4ecc03ce35 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:45.304553 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0fd58700-23eb-4b3c-9709-4c4ecc03ce35 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:45.305170 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0fd58700-23eb-4b3c-9709-4c4ecc03ce35 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:45.306037 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0fd58700-23eb-4b3c-9709-4c4ecc03ce35 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:45.306655 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0fd58700-23eb-4b3c-9709-4c4ecc03ce35 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:45.307332 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0fd58700-23eb-4b3c-9709-4c4ecc03ce35 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:45.478196 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-0fd58700-23eb-4b3c-9709-4c4ecc03ce35 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:45.493188 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-0fd58700-23eb-4b3c-9709-4c4ecc03ce35 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1321 microversion: 2.1 time: 0.203053 Oct 30 00:58:45.504525 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 136/282] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:45 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1321 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:45.719927 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-f3713fe0-2239-4bd8-b253-56d95b63e30c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:45.720584 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-f3713fe0-2239-4bd8-b253-56d95b63e30c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 6eaaf253-16e7-4c62-a434-a4b8f5d7b536] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:45.728090 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f3713fe0-2239-4bd8-b253-56d95b63e30c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:45.728574 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f3713fe0-2239-4bd8-b253-56d95b63e30c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:45.729007 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f3713fe0-2239-4bd8-b253-56d95b63e30c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:45.729701 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f3713fe0-2239-4bd8-b253-56d95b63e30c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:45.730150 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f3713fe0-2239-4bd8-b253-56d95b63e30c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:45.730569 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f3713fe0-2239-4bd8-b253-56d95b63e30c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:45.797755 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-f3713fe0-2239-4bd8-b253-56d95b63e30c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:45.808364 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-f3713fe0-2239-4bd8-b253-56d95b63e30c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536" status: 200 len: 1217 microversion: 2.1 time: 0.094998 Oct 30 00:58:45.815978 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 147/283] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:45 2022] GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536 => generated 1217 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:46.516535 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-3d2be2eb-3235-46a4-bb2e-f6081a8565b3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:46.517072 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-3d2be2eb-3235-46a4-bb2e-f6081a8565b3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:46.523537 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3d2be2eb-3235-46a4-bb2e-f6081a8565b3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:46.524055 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3d2be2eb-3235-46a4-bb2e-f6081a8565b3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:46.524485 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3d2be2eb-3235-46a4-bb2e-f6081a8565b3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:46.525182 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3d2be2eb-3235-46a4-bb2e-f6081a8565b3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:46.525630 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3d2be2eb-3235-46a4-bb2e-f6081a8565b3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:46.526047 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3d2be2eb-3235-46a4-bb2e-f6081a8565b3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:46.655425 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-3d2be2eb-3235-46a4-bb2e-f6081a8565b3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:46.664004 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-3d2be2eb-3235-46a4-bb2e-f6081a8565b3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1321 microversion: 2.1 time: 0.151713 Oct 30 00:58:46.664605 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 137/284] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:46 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1321 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:46.832799 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-2a55077b-e8db-4efc-918c-c17c672a5103 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:46.833161 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-2a55077b-e8db-4efc-918c-c17c672a5103 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 6eaaf253-16e7-4c62-a434-a4b8f5d7b536] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:46.839273 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2a55077b-e8db-4efc-918c-c17c672a5103 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:46.839745 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2a55077b-e8db-4efc-918c-c17c672a5103 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:46.840205 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2a55077b-e8db-4efc-918c-c17c672a5103 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:46.840905 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2a55077b-e8db-4efc-918c-c17c672a5103 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:46.841351 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2a55077b-e8db-4efc-918c-c17c672a5103 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:46.841768 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2a55077b-e8db-4efc-918c-c17c672a5103 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:46.906604 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-2a55077b-e8db-4efc-918c-c17c672a5103 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:46.914362 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-2a55077b-e8db-4efc-918c-c17c672a5103 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536" status: 200 len: 1217 microversion: 2.1 time: 0.085950 Oct 30 00:58:46.914923 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 148/285] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:46 2022] GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536 => generated 1217 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:47.687137 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-76062c18-3e52-4b3f-8fda-f78c6c8b2289 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:47.687889 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-76062c18-3e52-4b3f-8fda-f78c6c8b2289 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:47.696404 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-76062c18-3e52-4b3f-8fda-f78c6c8b2289 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:47.697101 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-76062c18-3e52-4b3f-8fda-f78c6c8b2289 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:47.697531 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-76062c18-3e52-4b3f-8fda-f78c6c8b2289 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:47.698727 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-76062c18-3e52-4b3f-8fda-f78c6c8b2289 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:47.699215 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-76062c18-3e52-4b3f-8fda-f78c6c8b2289 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:47.699628 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-76062c18-3e52-4b3f-8fda-f78c6c8b2289 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:47.856979 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-76062c18-3e52-4b3f-8fda-f78c6c8b2289 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:47.865041 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-76062c18-3e52-4b3f-8fda-f78c6c8b2289 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1321 microversion: 2.1 time: 0.182005 Oct 30 00:58:47.865866 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 138/286] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:47 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1321 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:47.939663 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-8aacbb8d-4177-4a80-ac72-461c526bf755 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:47.940722 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-8aacbb8d-4177-4a80-ac72-461c526bf755 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 6eaaf253-16e7-4c62-a434-a4b8f5d7b536] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:47.954417 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8aacbb8d-4177-4a80-ac72-461c526bf755 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:47.955197 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8aacbb8d-4177-4a80-ac72-461c526bf755 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:47.955846 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8aacbb8d-4177-4a80-ac72-461c526bf755 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:47.957053 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8aacbb8d-4177-4a80-ac72-461c526bf755 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:47.957616 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8aacbb8d-4177-4a80-ac72-461c526bf755 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:47.958211 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8aacbb8d-4177-4a80-ac72-461c526bf755 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:48.062805 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-8aacbb8d-4177-4a80-ac72-461c526bf755 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:48.073975 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-8aacbb8d-4177-4a80-ac72-461c526bf755 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536" status: 200 len: 1265 microversion: 2.1 time: 0.140270 Oct 30 00:58:48.075061 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 149/287] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:47 2022] GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536 => generated 1265 bytes in 142 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:48.131913 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [req-f46ff948-1301-4fa4-8755-fabdd67b691f req-a6e52e48-d14d-403e-b788-11be7031eb88 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c7be44ac-3b42-4ace-ac56-65c117e839c6", "name": "network-vif-plugged", "status": "completed", "tag": "2a5ef263-4503-4e15-8a9c-3e1f7cdcf464"}]} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:58:48.161177 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-f46ff948-1301-4fa4-8755-fabdd67b691f req-a6e52e48-d14d-403e-b788-11be7031eb88 service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:48.162341 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-f46ff948-1301-4fa4-8755-fabdd67b691f req-a6e52e48-d14d-403e-b788-11be7031eb88 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:48.162831 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-f46ff948-1301-4fa4-8755-fabdd67b691f req-a6e52e48-d14d-403e-b788-11be7031eb88 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:48.179099 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.compute.server_external_events [req-f46ff948-1301-4fa4-8755-fabdd67b691f req-a6e52e48-d14d-403e-b788-11be7031eb88 service nova] Creating event network-vif-plugged:2a5ef263-4503-4e15-8a9c-3e1f7cdcf464 for instance c7be44ac-3b42-4ace-ac56-65c117e839c6 on hv-cinder-maste Oct 30 00:58:48.186824 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [req-f46ff948-1301-4fa4-8755-fabdd67b691f req-a6e52e48-d14d-403e-b788-11be7031eb88 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.062671 Oct 30 00:58:48.194700 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 139/288] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 00:58:48 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:58:48.891246 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-d77b1414-3d46-48bb-901f-ed3b50566cb8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:48.891977 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-d77b1414-3d46-48bb-901f-ed3b50566cb8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:48.910543 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d77b1414-3d46-48bb-901f-ed3b50566cb8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:48.911044 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d77b1414-3d46-48bb-901f-ed3b50566cb8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:48.911478 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d77b1414-3d46-48bb-901f-ed3b50566cb8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:48.912442 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d77b1414-3d46-48bb-901f-ed3b50566cb8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:48.912888 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d77b1414-3d46-48bb-901f-ed3b50566cb8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:48.913316 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d77b1414-3d46-48bb-901f-ed3b50566cb8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:49.058895 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-d77b1414-3d46-48bb-901f-ed3b50566cb8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:49.066820 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-d77b1414-3d46-48bb-901f-ed3b50566cb8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1321 microversion: 2.1 time: 0.180039 Oct 30 00:58:49.067398 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 150/289] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:48 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1321 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:49.100337 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-a1fd2cfd-fab7-4de0-829c-f9ddba5b29a6 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:49.101053 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-a1fd2cfd-fab7-4de0-829c-f9ddba5b29a6 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 6eaaf253-16e7-4c62-a434-a4b8f5d7b536] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:49.107156 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a1fd2cfd-fab7-4de0-829c-f9ddba5b29a6 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:49.107630 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a1fd2cfd-fab7-4de0-829c-f9ddba5b29a6 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:49.108107 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a1fd2cfd-fab7-4de0-829c-f9ddba5b29a6 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:49.108769 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a1fd2cfd-fab7-4de0-829c-f9ddba5b29a6 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:49.109236 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a1fd2cfd-fab7-4de0-829c-f9ddba5b29a6 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:49.109677 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a1fd2cfd-fab7-4de0-829c-f9ddba5b29a6 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:49.244464 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-a1fd2cfd-fab7-4de0-829c-f9ddba5b29a6 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:49.253626 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-a1fd2cfd-fab7-4de0-829c-f9ddba5b29a6 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536" status: 200 len: 1313 microversion: 2.1 time: 0.158418 Oct 30 00:58:49.254222 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 140/290] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:49 2022] GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536 => generated 1313 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:49.962183 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [req-b2d8e360-f468-4d86-8a02-a5169e0e710e req-06b771bc-3442-4e0b-a564-f1ed629e48db service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6eaaf253-16e7-4c62-a434-a4b8f5d7b536", "tag": "789b3d8f-15d1-4b53-9724-f962176bd31c"}]} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:58:49.972642 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-b2d8e360-f468-4d86-8a02-a5169e0e710e req-06b771bc-3442-4e0b-a564-f1ed629e48db service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:49.973369 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-b2d8e360-f468-4d86-8a02-a5169e0e710e req-06b771bc-3442-4e0b-a564-f1ed629e48db service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:49.973984 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-b2d8e360-f468-4d86-8a02-a5169e0e710e req-06b771bc-3442-4e0b-a564-f1ed629e48db service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:49.989323 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.compute.server_external_events [req-b2d8e360-f468-4d86-8a02-a5169e0e710e req-06b771bc-3442-4e0b-a564-f1ed629e48db service nova] Creating event network-changed:789b3d8f-15d1-4b53-9724-f962176bd31c for instance 6eaaf253-16e7-4c62-a434-a4b8f5d7b536 on hv-cinder-maste Oct 30 00:58:50.001308 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [req-b2d8e360-f468-4d86-8a02-a5169e0e710e req-06b771bc-3442-4e0b-a564-f1ed629e48db service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042177 Oct 30 00:58:50.002601 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 151/291] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 00:58:49 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:58:50.090848 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-2c2dfb18-82a8-4b2a-a571-b1fb68bc3fce tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:50.092313 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-2c2dfb18-82a8-4b2a-a571-b1fb68bc3fce tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:50.099903 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2c2dfb18-82a8-4b2a-a571-b1fb68bc3fce tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:50.100766 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2c2dfb18-82a8-4b2a-a571-b1fb68bc3fce tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:50.102051 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2c2dfb18-82a8-4b2a-a571-b1fb68bc3fce tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:50.103237 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2c2dfb18-82a8-4b2a-a571-b1fb68bc3fce tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:50.103942 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2c2dfb18-82a8-4b2a-a571-b1fb68bc3fce tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:50.104616 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2c2dfb18-82a8-4b2a-a571-b1fb68bc3fce tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:50.279657 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-925de8cd-0c75-4680-8054-263f4cf17a99 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:50.280604 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-925de8cd-0c75-4680-8054-263f4cf17a99 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 6eaaf253-16e7-4c62-a434-a4b8f5d7b536] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:50.288163 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-925de8cd-0c75-4680-8054-263f4cf17a99 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:50.289202 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-925de8cd-0c75-4680-8054-263f4cf17a99 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:50.289706 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-925de8cd-0c75-4680-8054-263f4cf17a99 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:50.290516 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-925de8cd-0c75-4680-8054-263f4cf17a99 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:50.291012 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-925de8cd-0c75-4680-8054-263f4cf17a99 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:50.291506 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-925de8cd-0c75-4680-8054-263f4cf17a99 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:50.398380 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-2c2dfb18-82a8-4b2a-a571-b1fb68bc3fce tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:50.406591 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-2c2dfb18-82a8-4b2a-a571-b1fb68bc3fce tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1321 microversion: 2.1 time: 0.319559 Oct 30 00:58:50.419760 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 141/292] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:50 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1321 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:50.549397 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-925de8cd-0c75-4680-8054-263f4cf17a99 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:50.568164 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-925de8cd-0c75-4680-8054-263f4cf17a99 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536" status: 200 len: 1313 microversion: 2.1 time: 0.294779 Oct 30 00:58:50.573599 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 152/293] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:50 2022] GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536 => generated 1313 bytes in 301 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 00:58:51.431821 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-175b1498-9b40-400a-9f07-17c58cc42f6a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:51.432381 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-175b1498-9b40-400a-9f07-17c58cc42f6a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:51.434828 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_api:SELECT=86,nova_api:INSERT=9 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:58:51.441179 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-175b1498-9b40-400a-9f07-17c58cc42f6a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:51.441739 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-175b1498-9b40-400a-9f07-17c58cc42f6a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:51.442123 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-175b1498-9b40-400a-9f07-17c58cc42f6a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:51.442829 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-175b1498-9b40-400a-9f07-17c58cc42f6a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:51.443278 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-175b1498-9b40-400a-9f07-17c58cc42f6a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:51.444041 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-175b1498-9b40-400a-9f07-17c58cc42f6a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:51.588566 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-175b1498-9b40-400a-9f07-17c58cc42f6a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:51.601509 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-508c1694-b6b4-43c1-80c1-8f782a332c96 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:51.602053 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-508c1694-b6b4-43c1-80c1-8f782a332c96 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 6eaaf253-16e7-4c62-a434-a4b8f5d7b536] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:51.605111 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_api:SELECT=78,nova_api:INSERT=3 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:58:51.609697 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-175b1498-9b40-400a-9f07-17c58cc42f6a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1321 microversion: 2.1 time: 0.181876 Oct 30 00:58:51.610275 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 142/294] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:51 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1321 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:51.612221 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-508c1694-b6b4-43c1-80c1-8f782a332c96 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:51.612890 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-508c1694-b6b4-43c1-80c1-8f782a332c96 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:51.613475 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-508c1694-b6b4-43c1-80c1-8f782a332c96 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:51.614453 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-508c1694-b6b4-43c1-80c1-8f782a332c96 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:51.615046 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-508c1694-b6b4-43c1-80c1-8f782a332c96 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:51.615626 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-508c1694-b6b4-43c1-80c1-8f782a332c96 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:51.757917 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-508c1694-b6b4-43c1-80c1-8f782a332c96 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:51.766050 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-508c1694-b6b4-43c1-80c1-8f782a332c96 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536" status: 200 len: 1313 microversion: 2.1 time: 0.168680 Oct 30 00:58:51.766671 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 153/295] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:51 2022] GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536 => generated 1313 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 00:58:52.632766 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-d553f4ff-904d-4429-9792-c7280222fa5a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:52.633422 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-d553f4ff-904d-4429-9792-c7280222fa5a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:52.640085 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d553f4ff-904d-4429-9792-c7280222fa5a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:52.640565 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d553f4ff-904d-4429-9792-c7280222fa5a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:52.640991 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d553f4ff-904d-4429-9792-c7280222fa5a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:52.641788 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d553f4ff-904d-4429-9792-c7280222fa5a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:52.642234 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d553f4ff-904d-4429-9792-c7280222fa5a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:52.642655 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d553f4ff-904d-4429-9792-c7280222fa5a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:52.787760 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-d553f4ff-904d-4429-9792-c7280222fa5a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:52.795916 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-d553f4ff-904d-4429-9792-c7280222fa5a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1321 microversion: 2.1 time: 0.167160 Oct 30 00:58:52.796087 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-bdfccab6-a7cd-4308-a49b-30a33442166f tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:52.796566 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-bdfccab6-a7cd-4308-a49b-30a33442166f tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 6eaaf253-16e7-4c62-a434-a4b8f5d7b536] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:52.801546 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 143/296] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:52 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1321 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:52.802698 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bdfccab6-a7cd-4308-a49b-30a33442166f tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:52.803194 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bdfccab6-a7cd-4308-a49b-30a33442166f tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:52.803630 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bdfccab6-a7cd-4308-a49b-30a33442166f tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:52.804365 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bdfccab6-a7cd-4308-a49b-30a33442166f tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:52.804810 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bdfccab6-a7cd-4308-a49b-30a33442166f tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:52.805231 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bdfccab6-a7cd-4308-a49b-30a33442166f tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:52.938008 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-bdfccab6-a7cd-4308-a49b-30a33442166f tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:52.946483 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-bdfccab6-a7cd-4308-a49b-30a33442166f tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536" status: 200 len: 1313 microversion: 2.1 time: 0.154374 Oct 30 00:58:52.947055 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 154/297] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:52 2022] GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536 => generated 1313 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:53.820797 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-ae292f63-dd8b-4804-9d42-6642a7fe9e2c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:53.821504 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-ae292f63-dd8b-4804-9d42-6642a7fe9e2c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:53.829988 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ae292f63-dd8b-4804-9d42-6642a7fe9e2c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:53.830696 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ae292f63-dd8b-4804-9d42-6642a7fe9e2c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:53.831185 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ae292f63-dd8b-4804-9d42-6642a7fe9e2c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:53.832103 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ae292f63-dd8b-4804-9d42-6642a7fe9e2c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:53.832726 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ae292f63-dd8b-4804-9d42-6642a7fe9e2c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:53.833318 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ae292f63-dd8b-4804-9d42-6642a7fe9e2c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:53.970147 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-529de02f-22a9-4e5d-8ecc-0e8e908e691c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:53.970843 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-529de02f-22a9-4e5d-8ecc-0e8e908e691c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 6eaaf253-16e7-4c62-a434-a4b8f5d7b536] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:53.977063 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-529de02f-22a9-4e5d-8ecc-0e8e908e691c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:53.977699 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-529de02f-22a9-4e5d-8ecc-0e8e908e691c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:53.978216 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-529de02f-22a9-4e5d-8ecc-0e8e908e691c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:53.978975 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-529de02f-22a9-4e5d-8ecc-0e8e908e691c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:53.979584 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-529de02f-22a9-4e5d-8ecc-0e8e908e691c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:53.980169 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-529de02f-22a9-4e5d-8ecc-0e8e908e691c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:53.986921 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-ae292f63-dd8b-4804-9d42-6642a7fe9e2c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:53.995297 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-ae292f63-dd8b-4804-9d42-6642a7fe9e2c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1321 microversion: 2.1 time: 0.180305 Oct 30 00:58:53.996474 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 144/298] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:53 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1321 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:54.151794 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-529de02f-22a9-4e5d-8ecc-0e8e908e691c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:54.171954 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-529de02f-22a9-4e5d-8ecc-0e8e908e691c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536" status: 200 len: 1313 microversion: 2.1 time: 0.194557 Oct 30 00:58:54.172737 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 155/299] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:53 2022] GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536 => generated 1313 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 00:58:55.019109 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-730b1858-be18-417e-af71-9e44469f1504 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:55.019742 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-730b1858-be18-417e-af71-9e44469f1504 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:55.021629 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell0:SELECT=2 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:58:55.022299 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell0:SELECT=6 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:58:55.029250 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-730b1858-be18-417e-af71-9e44469f1504 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:55.029750 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-730b1858-be18-417e-af71-9e44469f1504 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:55.030213 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-730b1858-be18-417e-af71-9e44469f1504 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:55.031146 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-730b1858-be18-417e-af71-9e44469f1504 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:55.031600 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-730b1858-be18-417e-af71-9e44469f1504 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:55.032246 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-730b1858-be18-417e-af71-9e44469f1504 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:55.177906 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-730b1858-be18-417e-af71-9e44469f1504 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:55.189130 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-730b1858-be18-417e-af71-9e44469f1504 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1321 microversion: 2.1 time: 0.174319 Oct 30 00:58:55.189720 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 145/300] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:55 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1321 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:55.196354 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-42a3c499-df0f-4930-a648-25ff7e77d974 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:55.196971 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-42a3c499-df0f-4930-a648-25ff7e77d974 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 6eaaf253-16e7-4c62-a434-a4b8f5d7b536] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:55.199194 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell0:SELECT=2 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:58:55.205670 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-42a3c499-df0f-4930-a648-25ff7e77d974 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:55.206372 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-42a3c499-df0f-4930-a648-25ff7e77d974 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:55.206994 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-42a3c499-df0f-4930-a648-25ff7e77d974 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:55.208135 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-42a3c499-df0f-4930-a648-25ff7e77d974 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:55.208821 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-42a3c499-df0f-4930-a648-25ff7e77d974 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:55.209454 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-42a3c499-df0f-4930-a648-25ff7e77d974 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:55.357710 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-42a3c499-df0f-4930-a648-25ff7e77d974 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:55.366190 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-42a3c499-df0f-4930-a648-25ff7e77d974 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536" status: 200 len: 1313 microversion: 2.1 time: 0.173737 Oct 30 00:58:55.366772 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 156/301] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:55 2022] GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536 => generated 1313 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:56.212502 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-67360662-6c50-4476-a013-4324eb286e5d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:56.213110 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-67360662-6c50-4476-a013-4324eb286e5d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:56.220083 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-67360662-6c50-4476-a013-4324eb286e5d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:56.220565 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-67360662-6c50-4476-a013-4324eb286e5d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:56.220988 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-67360662-6c50-4476-a013-4324eb286e5d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:56.221682 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-67360662-6c50-4476-a013-4324eb286e5d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:56.222123 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-67360662-6c50-4476-a013-4324eb286e5d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:56.222534 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-67360662-6c50-4476-a013-4324eb286e5d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:56.360796 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-67360662-6c50-4476-a013-4324eb286e5d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:56.369104 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-67360662-6c50-4476-a013-4324eb286e5d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1321 microversion: 2.1 time: 0.160555 Oct 30 00:58:56.369714 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 146/302] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:56 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1321 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:56.389640 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-9b8836b3-318b-48c7-a45c-0d3041ca1bf1 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:56.390223 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-9b8836b3-318b-48c7-a45c-0d3041ca1bf1 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 6eaaf253-16e7-4c62-a434-a4b8f5d7b536] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:56.396893 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9b8836b3-318b-48c7-a45c-0d3041ca1bf1 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:56.397371 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9b8836b3-318b-48c7-a45c-0d3041ca1bf1 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:56.397823 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9b8836b3-318b-48c7-a45c-0d3041ca1bf1 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:56.398482 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9b8836b3-318b-48c7-a45c-0d3041ca1bf1 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:56.398924 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9b8836b3-318b-48c7-a45c-0d3041ca1bf1 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:56.399368 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9b8836b3-318b-48c7-a45c-0d3041ca1bf1 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:56.534926 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-9b8836b3-318b-48c7-a45c-0d3041ca1bf1 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:56.543378 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-9b8836b3-318b-48c7-a45c-0d3041ca1bf1 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536" status: 200 len: 1313 microversion: 2.1 time: 0.157585 Oct 30 00:58:56.544069 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 157/303] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:56 2022] GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536 => generated 1313 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:57.392177 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-878fb989-c815-4635-93da-b6714cf9d2d7 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:57.392894 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-878fb989-c815-4635-93da-b6714cf9d2d7 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:57.400363 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-878fb989-c815-4635-93da-b6714cf9d2d7 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:57.400756 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-878fb989-c815-4635-93da-b6714cf9d2d7 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:57.401213 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-878fb989-c815-4635-93da-b6714cf9d2d7 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:57.401938 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-878fb989-c815-4635-93da-b6714cf9d2d7 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:57.402425 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-878fb989-c815-4635-93da-b6714cf9d2d7 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:57.402841 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-878fb989-c815-4635-93da-b6714cf9d2d7 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:57.546211 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-878fb989-c815-4635-93da-b6714cf9d2d7 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:57.554068 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-878fb989-c815-4635-93da-b6714cf9d2d7 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1321 microversion: 2.1 time: 0.165891 Oct 30 00:58:57.555022 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 147/304] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:57 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1321 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:57.570990 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-7790a841-55ac-4946-8dc7-29d78f864b0d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:57.571572 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-7790a841-55ac-4946-8dc7-29d78f864b0d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 6eaaf253-16e7-4c62-a434-a4b8f5d7b536] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:57.578330 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7790a841-55ac-4946-8dc7-29d78f864b0d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:57.578807 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7790a841-55ac-4946-8dc7-29d78f864b0d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:57.579364 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7790a841-55ac-4946-8dc7-29d78f864b0d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:57.580069 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7790a841-55ac-4946-8dc7-29d78f864b0d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:57.580727 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7790a841-55ac-4946-8dc7-29d78f864b0d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:57.580923 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7790a841-55ac-4946-8dc7-29d78f864b0d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:57.750607 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-7790a841-55ac-4946-8dc7-29d78f864b0d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:57.758866 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-7790a841-55ac-4946-8dc7-29d78f864b0d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536" status: 200 len: 1313 microversion: 2.1 time: 0.191882 Oct 30 00:58:57.759897 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 158/305] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:57 2022] GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536 => generated 1313 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:58.577914 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-2353e6d7-afd2-4e36-a3aa-4c7a7832e1b6 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:58.578494 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-2353e6d7-afd2-4e36-a3aa-4c7a7832e1b6 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:58.585769 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2353e6d7-afd2-4e36-a3aa-4c7a7832e1b6 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:58.586252 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2353e6d7-afd2-4e36-a3aa-4c7a7832e1b6 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:58.586690 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2353e6d7-afd2-4e36-a3aa-4c7a7832e1b6 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:58.587390 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2353e6d7-afd2-4e36-a3aa-4c7a7832e1b6 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:58.587885 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2353e6d7-afd2-4e36-a3aa-4c7a7832e1b6 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:58.588305 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2353e6d7-afd2-4e36-a3aa-4c7a7832e1b6 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:58.733139 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-2353e6d7-afd2-4e36-a3aa-4c7a7832e1b6 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:58.741229 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-2353e6d7-afd2-4e36-a3aa-4c7a7832e1b6 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1321 microversion: 2.1 time: 0.167796 Oct 30 00:58:58.742263 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 148/306] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:58 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1321 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:58.782769 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-9d8f6f8b-d212-4384-be56-351037779311 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:58.783521 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-9d8f6f8b-d212-4384-be56-351037779311 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 6eaaf253-16e7-4c62-a434-a4b8f5d7b536] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:58.789876 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9d8f6f8b-d212-4384-be56-351037779311 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:58.790362 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9d8f6f8b-d212-4384-be56-351037779311 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:58.790793 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9d8f6f8b-d212-4384-be56-351037779311 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:58.791482 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9d8f6f8b-d212-4384-be56-351037779311 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:58.792093 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9d8f6f8b-d212-4384-be56-351037779311 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:58.792514 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9d8f6f8b-d212-4384-be56-351037779311 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:58.929145 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-9d8f6f8b-d212-4384-be56-351037779311 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:58.940176 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-9d8f6f8b-d212-4384-be56-351037779311 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536" status: 200 len: 1313 microversion: 2.1 time: 0.161628 Oct 30 00:58:58.940757 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 159/307] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:58 2022] GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536 => generated 1313 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:59.766181 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-48515670-7a6d-492e-8c38-8baa72cbd44d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:59.767195 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-48515670-7a6d-492e-8c38-8baa72cbd44d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:59.775806 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-48515670-7a6d-492e-8c38-8baa72cbd44d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:59.776449 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-48515670-7a6d-492e-8c38-8baa72cbd44d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:59.777049 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-48515670-7a6d-492e-8c38-8baa72cbd44d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:59.777947 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-48515670-7a6d-492e-8c38-8baa72cbd44d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:59.778515 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-48515670-7a6d-492e-8c38-8baa72cbd44d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:59.779066 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-48515670-7a6d-492e-8c38-8baa72cbd44d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:59.922313 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-48515670-7a6d-492e-8c38-8baa72cbd44d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:58:59.930389 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-48515670-7a6d-492e-8c38-8baa72cbd44d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1321 microversion: 2.1 time: 0.169322 Oct 30 00:58:59.930946 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 149/308] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:59 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1321 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:58:59.966309 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-f8608d46-d83d-4b23-99cd-9e8284925c71 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:58:59.966890 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-f8608d46-d83d-4b23-99cd-9e8284925c71 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 6eaaf253-16e7-4c62-a434-a4b8f5d7b536] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:58:59.974998 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f8608d46-d83d-4b23-99cd-9e8284925c71 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:59.975495 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f8608d46-d83d-4b23-99cd-9e8284925c71 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:59.975960 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f8608d46-d83d-4b23-99cd-9e8284925c71 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:58:59.976654 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f8608d46-d83d-4b23-99cd-9e8284925c71 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:58:59.977115 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f8608d46-d83d-4b23-99cd-9e8284925c71 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:58:59.977537 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f8608d46-d83d-4b23-99cd-9e8284925c71 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:00.122861 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-f8608d46-d83d-4b23-99cd-9e8284925c71 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:00.131276 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-f8608d46-d83d-4b23-99cd-9e8284925c71 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536" status: 200 len: 1313 microversion: 2.1 time: 0.171306 Oct 30 00:59:00.131867 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 160/309] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:58:59 2022] GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536 => generated 1313 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:00.956895 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-bf076814-60da-4b5f-9066-29a9e716db9d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:00.957483 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-bf076814-60da-4b5f-9066-29a9e716db9d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:00.964429 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bf076814-60da-4b5f-9066-29a9e716db9d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:00.964902 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bf076814-60da-4b5f-9066-29a9e716db9d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:00.965328 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bf076814-60da-4b5f-9066-29a9e716db9d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:00.966040 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bf076814-60da-4b5f-9066-29a9e716db9d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:00.966481 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bf076814-60da-4b5f-9066-29a9e716db9d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:00.966894 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bf076814-60da-4b5f-9066-29a9e716db9d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:01.113806 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-bf076814-60da-4b5f-9066-29a9e716db9d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:01.124406 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-bf076814-60da-4b5f-9066-29a9e716db9d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1321 microversion: 2.1 time: 0.172838 Oct 30 00:59:01.125172 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 150/310] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:00 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1321 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:01.156848 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-cadda74d-78a3-4468-b86b-429e04294af7 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:01.157533 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-cadda74d-78a3-4468-b86b-429e04294af7 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 6eaaf253-16e7-4c62-a434-a4b8f5d7b536] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:01.164024 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cadda74d-78a3-4468-b86b-429e04294af7 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:01.164505 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cadda74d-78a3-4468-b86b-429e04294af7 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:01.164932 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cadda74d-78a3-4468-b86b-429e04294af7 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:01.165618 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cadda74d-78a3-4468-b86b-429e04294af7 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:01.166059 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cadda74d-78a3-4468-b86b-429e04294af7 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:01.166472 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cadda74d-78a3-4468-b86b-429e04294af7 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:01.305463 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-cadda74d-78a3-4468-b86b-429e04294af7 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:01.316699 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-cadda74d-78a3-4468-b86b-429e04294af7 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536" status: 200 len: 1313 microversion: 2.1 time: 0.163756 Oct 30 00:59:01.317253 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 161/311] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:01 2022] GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536 => generated 1313 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:02.150939 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-0054c45d-39d8-4e93-a095-cbb215c19a70 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:02.151958 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-0054c45d-39d8-4e93-a095-cbb215c19a70 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:02.162557 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0054c45d-39d8-4e93-a095-cbb215c19a70 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:02.163349 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0054c45d-39d8-4e93-a095-cbb215c19a70 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:02.164127 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0054c45d-39d8-4e93-a095-cbb215c19a70 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:02.165258 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0054c45d-39d8-4e93-a095-cbb215c19a70 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:02.165970 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0054c45d-39d8-4e93-a095-cbb215c19a70 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:02.166715 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0054c45d-39d8-4e93-a095-cbb215c19a70 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:02.305431 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-0054c45d-39d8-4e93-a095-cbb215c19a70 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:02.313138 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-0054c45d-39d8-4e93-a095-cbb215c19a70 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1321 microversion: 2.1 time: 0.169092 Oct 30 00:59:02.313723 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 151/312] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:02 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1321 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:02.340091 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-92dc5a2e-2296-42d5-8d07-3259cfb12dad tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:02.340694 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-92dc5a2e-2296-42d5-8d07-3259cfb12dad tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 6eaaf253-16e7-4c62-a434-a4b8f5d7b536] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:02.347130 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-92dc5a2e-2296-42d5-8d07-3259cfb12dad tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:02.347610 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-92dc5a2e-2296-42d5-8d07-3259cfb12dad tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:02.348063 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-92dc5a2e-2296-42d5-8d07-3259cfb12dad tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:02.348749 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-92dc5a2e-2296-42d5-8d07-3259cfb12dad tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:02.349194 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-92dc5a2e-2296-42d5-8d07-3259cfb12dad tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:02.349601 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-92dc5a2e-2296-42d5-8d07-3259cfb12dad tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:02.487924 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-92dc5a2e-2296-42d5-8d07-3259cfb12dad tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:02.499782 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-92dc5a2e-2296-42d5-8d07-3259cfb12dad tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536" status: 200 len: 1313 microversion: 2.1 time: 0.163222 Oct 30 00:59:02.500589 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 162/313] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:02 2022] GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536 => generated 1313 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:03.337110 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-a913ed3a-a997-4715-9f34-273d69e79acf tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:03.337766 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-a913ed3a-a997-4715-9f34-273d69e79acf tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:03.344807 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a913ed3a-a997-4715-9f34-273d69e79acf tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:03.345307 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a913ed3a-a997-4715-9f34-273d69e79acf tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:03.345799 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a913ed3a-a997-4715-9f34-273d69e79acf tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:03.346720 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a913ed3a-a997-4715-9f34-273d69e79acf tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:03.347207 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a913ed3a-a997-4715-9f34-273d69e79acf tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:03.347623 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a913ed3a-a997-4715-9f34-273d69e79acf tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:03.480309 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-a913ed3a-a997-4715-9f34-273d69e79acf tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:03.487997 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-a913ed3a-a997-4715-9f34-273d69e79acf tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1321 microversion: 2.1 time: 0.154889 Oct 30 00:59:03.488613 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 152/314] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:03 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1321 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:03.523505 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-b157038f-add6-4d8c-9f7d-60adf99fd209 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:03.524059 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-b157038f-add6-4d8c-9f7d-60adf99fd209 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 6eaaf253-16e7-4c62-a434-a4b8f5d7b536] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:03.530276 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b157038f-add6-4d8c-9f7d-60adf99fd209 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:03.530772 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b157038f-add6-4d8c-9f7d-60adf99fd209 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:03.531196 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b157038f-add6-4d8c-9f7d-60adf99fd209 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:03.531924 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b157038f-add6-4d8c-9f7d-60adf99fd209 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:03.532372 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b157038f-add6-4d8c-9f7d-60adf99fd209 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:03.532780 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b157038f-add6-4d8c-9f7d-60adf99fd209 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:03.671081 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-b157038f-add6-4d8c-9f7d-60adf99fd209 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:03.674557 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell1:SELECT=100 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:59:03.680324 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-b157038f-add6-4d8c-9f7d-60adf99fd209 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536" status: 200 len: 1313 microversion: 2.1 time: 0.160858 Oct 30 00:59:03.680887 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 163/315] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:03 2022] GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536 => generated 1313 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:04.511517 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-6fcdbcef-f45b-4bb5-b9d0-de863111d986 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:04.512152 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-6fcdbcef-f45b-4bb5-b9d0-de863111d986 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:04.518145 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6fcdbcef-f45b-4bb5-b9d0-de863111d986 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:04.518698 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6fcdbcef-f45b-4bb5-b9d0-de863111d986 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:04.519132 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6fcdbcef-f45b-4bb5-b9d0-de863111d986 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:04.519856 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6fcdbcef-f45b-4bb5-b9d0-de863111d986 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:04.520310 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6fcdbcef-f45b-4bb5-b9d0-de863111d986 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:04.520936 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6fcdbcef-f45b-4bb5-b9d0-de863111d986 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:04.525035 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell1:SELECT=100 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:59:04.593250 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [req-f1bab57c-ead2-43a8-876a-390372689ac5 req-e1bc7435-c1cf-4a46-a757-e111d061c836 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "60b2f6e0-a160-4d3f-b036-38597b60da68", "name": "network-vif-plugged", "status": "completed", "tag": "88fd74d2-892d-4aaf-b602-415c64c5b9aa"}]} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:59:04.605832 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-f1bab57c-ead2-43a8-876a-390372689ac5 req-e1bc7435-c1cf-4a46-a757-e111d061c836 service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:04.606315 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-f1bab57c-ead2-43a8-876a-390372689ac5 req-e1bc7435-c1cf-4a46-a757-e111d061c836 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:04.606748 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-f1bab57c-ead2-43a8-876a-390372689ac5 req-e1bc7435-c1cf-4a46-a757-e111d061c836 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:04.618649 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.compute.server_external_events [req-f1bab57c-ead2-43a8-876a-390372689ac5 req-e1bc7435-c1cf-4a46-a757-e111d061c836 service nova] Creating event network-vif-plugged:88fd74d2-892d-4aaf-b602-415c64c5b9aa for instance 60b2f6e0-a160-4d3f-b036-38597b60da68 on hv-cinder-maste Oct 30 00:59:04.624151 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [req-f1bab57c-ead2-43a8-876a-390372689ac5 req-e1bc7435-c1cf-4a46-a757-e111d061c836 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035084 Oct 30 00:59:04.624903 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 164/316] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 00:59:04 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:59:04.656975 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-6fcdbcef-f45b-4bb5-b9d0-de863111d986 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:04.666338 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-6fcdbcef-f45b-4bb5-b9d0-de863111d986 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1321 microversion: 2.1 time: 0.159194 Oct 30 00:59:04.667343 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 153/317] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:04 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1321 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:04.704401 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-c88feb24-0367-43bd-8072-5680001f9773 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:04.705070 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-c88feb24-0367-43bd-8072-5680001f9773 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 6eaaf253-16e7-4c62-a434-a4b8f5d7b536] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:04.711451 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c88feb24-0367-43bd-8072-5680001f9773 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:04.711970 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c88feb24-0367-43bd-8072-5680001f9773 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:04.712403 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c88feb24-0367-43bd-8072-5680001f9773 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:04.713091 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c88feb24-0367-43bd-8072-5680001f9773 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:04.713541 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c88feb24-0367-43bd-8072-5680001f9773 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:04.713967 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c88feb24-0367-43bd-8072-5680001f9773 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:04.843223 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-c88feb24-0367-43bd-8072-5680001f9773 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:04.851252 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-c88feb24-0367-43bd-8072-5680001f9773 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536" status: 200 len: 1313 microversion: 2.1 time: 0.151151 Oct 30 00:59:04.851899 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 165/318] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:04 2022] GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536 => generated 1313 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:05.693976 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-a380ba7f-3cc0-4e7b-9240-4de8f7ee6f94 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:05.694582 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-a380ba7f-3cc0-4e7b-9240-4de8f7ee6f94 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:05.701917 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a380ba7f-3cc0-4e7b-9240-4de8f7ee6f94 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:05.702400 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a380ba7f-3cc0-4e7b-9240-4de8f7ee6f94 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:05.702832 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a380ba7f-3cc0-4e7b-9240-4de8f7ee6f94 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:05.703526 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a380ba7f-3cc0-4e7b-9240-4de8f7ee6f94 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:05.704012 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a380ba7f-3cc0-4e7b-9240-4de8f7ee6f94 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:05.704433 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a380ba7f-3cc0-4e7b-9240-4de8f7ee6f94 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:05.839330 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-a380ba7f-3cc0-4e7b-9240-4de8f7ee6f94 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:05.849016 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-a380ba7f-3cc0-4e7b-9240-4de8f7ee6f94 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1321 microversion: 2.1 time: 0.159250 Oct 30 00:59:05.849825 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 154/319] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:05 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1321 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:05.874358 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-9a3735fc-b782-432a-8d75-39d410e75166 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:05.874963 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-9a3735fc-b782-432a-8d75-39d410e75166 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 6eaaf253-16e7-4c62-a434-a4b8f5d7b536] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:05.882011 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9a3735fc-b782-432a-8d75-39d410e75166 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:05.882491 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9a3735fc-b782-432a-8d75-39d410e75166 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:05.882915 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9a3735fc-b782-432a-8d75-39d410e75166 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:05.883608 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9a3735fc-b782-432a-8d75-39d410e75166 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:05.884124 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9a3735fc-b782-432a-8d75-39d410e75166 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:05.884538 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9a3735fc-b782-432a-8d75-39d410e75166 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:06.017785 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-9a3735fc-b782-432a-8d75-39d410e75166 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:06.025990 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-9a3735fc-b782-432a-8d75-39d410e75166 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536" status: 200 len: 1313 microversion: 2.1 time: 0.155465 Oct 30 00:59:06.026562 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 166/320] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:05 2022] GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536 => generated 1313 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:06.881203 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-99c7cd5f-30b0-4fbd-a414-dfd7965feed0 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:06.882224 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-99c7cd5f-30b0-4fbd-a414-dfd7965feed0 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:06.893182 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-99c7cd5f-30b0-4fbd-a414-dfd7965feed0 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:06.893979 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-99c7cd5f-30b0-4fbd-a414-dfd7965feed0 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:06.894706 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-99c7cd5f-30b0-4fbd-a414-dfd7965feed0 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:06.896092 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-99c7cd5f-30b0-4fbd-a414-dfd7965feed0 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:06.896555 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-99c7cd5f-30b0-4fbd-a414-dfd7965feed0 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:06.896978 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-99c7cd5f-30b0-4fbd-a414-dfd7965feed0 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:07.032075 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-99c7cd5f-30b0-4fbd-a414-dfd7965feed0 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:07.040196 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-99c7cd5f-30b0-4fbd-a414-dfd7965feed0 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1321 microversion: 2.1 time: 0.165771 Oct 30 00:59:07.040698 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 155/321] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:06 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1321 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:07.049167 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-ddeb4690-f04a-435d-a13a-d3682759cde3 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:07.049768 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-ddeb4690-f04a-435d-a13a-d3682759cde3 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 6eaaf253-16e7-4c62-a434-a4b8f5d7b536] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:07.055873 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ddeb4690-f04a-435d-a13a-d3682759cde3 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:07.056356 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ddeb4690-f04a-435d-a13a-d3682759cde3 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:07.056835 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ddeb4690-f04a-435d-a13a-d3682759cde3 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:07.057499 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ddeb4690-f04a-435d-a13a-d3682759cde3 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:07.057947 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ddeb4690-f04a-435d-a13a-d3682759cde3 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:07.058366 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ddeb4690-f04a-435d-a13a-d3682759cde3 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:07.194130 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-ddeb4690-f04a-435d-a13a-d3682759cde3 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:07.202744 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-ddeb4690-f04a-435d-a13a-d3682759cde3 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536" status: 200 len: 1313 microversion: 2.1 time: 0.157677 Oct 30 00:59:07.203341 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 167/322] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:07 2022] GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536 => generated 1313 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:08.067300 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-85ae2ead-fbfd-4858-a431-45673cb578b4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:08.068334 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-85ae2ead-fbfd-4858-a431-45673cb578b4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:08.079851 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-85ae2ead-fbfd-4858-a431-45673cb578b4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:08.080640 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-85ae2ead-fbfd-4858-a431-45673cb578b4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:08.081364 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-85ae2ead-fbfd-4858-a431-45673cb578b4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:08.082548 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-85ae2ead-fbfd-4858-a431-45673cb578b4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:08.083299 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-85ae2ead-fbfd-4858-a431-45673cb578b4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:08.084069 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-85ae2ead-fbfd-4858-a431-45673cb578b4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:08.226863 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-70ca35b0-d6f3-4b7b-9cef-29ec315de168 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:08.227481 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-70ca35b0-d6f3-4b7b-9cef-29ec315de168 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 6eaaf253-16e7-4c62-a434-a4b8f5d7b536] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:08.233709 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-85ae2ead-fbfd-4858-a431-45673cb578b4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:08.233883 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-70ca35b0-d6f3-4b7b-9cef-29ec315de168 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:08.234206 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-70ca35b0-d6f3-4b7b-9cef-29ec315de168 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:08.234636 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-70ca35b0-d6f3-4b7b-9cef-29ec315de168 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:08.235323 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-70ca35b0-d6f3-4b7b-9cef-29ec315de168 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:08.235981 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-70ca35b0-d6f3-4b7b-9cef-29ec315de168 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:08.236371 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-70ca35b0-d6f3-4b7b-9cef-29ec315de168 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:08.241804 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-85ae2ead-fbfd-4858-a431-45673cb578b4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1321 microversion: 2.1 time: 0.181033 Oct 30 00:59:08.242379 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 156/323] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:08 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1321 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:08.368783 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-70ca35b0-d6f3-4b7b-9cef-29ec315de168 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:08.376958 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-70ca35b0-d6f3-4b7b-9cef-29ec315de168 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536" status: 200 len: 1313 microversion: 2.1 time: 0.154179 Oct 30 00:59:08.377523 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 168/324] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:08 2022] GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536 => generated 1313 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:09.264309 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-d1730ad0-17a4-4e79-a1f0-e3912b9882b7 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:09.264949 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-d1730ad0-17a4-4e79-a1f0-e3912b9882b7 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:09.271679 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d1730ad0-17a4-4e79-a1f0-e3912b9882b7 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:09.272240 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d1730ad0-17a4-4e79-a1f0-e3912b9882b7 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:09.272666 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d1730ad0-17a4-4e79-a1f0-e3912b9882b7 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:09.273334 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d1730ad0-17a4-4e79-a1f0-e3912b9882b7 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:09.273778 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d1730ad0-17a4-4e79-a1f0-e3912b9882b7 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:09.274193 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d1730ad0-17a4-4e79-a1f0-e3912b9882b7 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:09.400356 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-6bca0633-6add-4803-ad7f-ed45825c721a tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:09.400947 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-6bca0633-6add-4803-ad7f-ed45825c721a tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 6eaaf253-16e7-4c62-a434-a4b8f5d7b536] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:09.404337 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-d1730ad0-17a4-4e79-a1f0-e3912b9882b7 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:09.407438 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6bca0633-6add-4803-ad7f-ed45825c721a tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:09.407956 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6bca0633-6add-4803-ad7f-ed45825c721a tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:09.408380 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6bca0633-6add-4803-ad7f-ed45825c721a tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:09.409110 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6bca0633-6add-4803-ad7f-ed45825c721a tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:09.409554 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6bca0633-6add-4803-ad7f-ed45825c721a tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:09.409967 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6bca0633-6add-4803-ad7f-ed45825c721a tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:09.412777 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-d1730ad0-17a4-4e79-a1f0-e3912b9882b7 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1321 microversion: 2.1 time: 0.152238 Oct 30 00:59:09.413611 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 157/325] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:09 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1321 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:09.545734 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-6bca0633-6add-4803-ad7f-ed45825c721a tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:09.553432 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-6bca0633-6add-4803-ad7f-ed45825c721a tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536" status: 200 len: 1313 microversion: 2.1 time: 0.156896 Oct 30 00:59:09.553998 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 169/326] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:09 2022] GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536 => generated 1313 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:10.440000 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-ca8f85ce-fb98-4aaf-9d64-db7dce555d36 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:10.441053 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-ca8f85ce-fb98-4aaf-9d64-db7dce555d36 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:10.452716 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ca8f85ce-fb98-4aaf-9d64-db7dce555d36 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:10.453526 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ca8f85ce-fb98-4aaf-9d64-db7dce555d36 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:10.454251 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ca8f85ce-fb98-4aaf-9d64-db7dce555d36 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:10.455431 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ca8f85ce-fb98-4aaf-9d64-db7dce555d36 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:10.456201 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ca8f85ce-fb98-4aaf-9d64-db7dce555d36 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:10.456951 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ca8f85ce-fb98-4aaf-9d64-db7dce555d36 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:10.577789 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-adb735e5-687c-4747-b388-5325fab63491 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:10.578741 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-adb735e5-687c-4747-b388-5325fab63491 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 6eaaf253-16e7-4c62-a434-a4b8f5d7b536] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:10.585320 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-adb735e5-687c-4747-b388-5325fab63491 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:10.586059 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-adb735e5-687c-4747-b388-5325fab63491 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:10.586500 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-adb735e5-687c-4747-b388-5325fab63491 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:10.587204 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-adb735e5-687c-4747-b388-5325fab63491 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:10.587673 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-adb735e5-687c-4747-b388-5325fab63491 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:10.588128 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-adb735e5-687c-4747-b388-5325fab63491 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:10.590221 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-ca8f85ce-fb98-4aaf-9d64-db7dce555d36 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:10.597769 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-ca8f85ce-fb98-4aaf-9d64-db7dce555d36 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1321 microversion: 2.1 time: 0.164361 Oct 30 00:59:10.598320 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 158/327] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:10 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1321 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:10.722746 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-adb735e5-687c-4747-b388-5325fab63491 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:10.731117 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-adb735e5-687c-4747-b388-5325fab63491 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536" status: 200 len: 1313 microversion: 2.1 time: 0.158009 Oct 30 00:59:10.732050 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 170/328] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:10 2022] GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536 => generated 1313 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:11.626885 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-0edda527-c668-4195-b705-3f76ee04f851 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:11.627683 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-0edda527-c668-4195-b705-3f76ee04f851 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:11.636205 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0edda527-c668-4195-b705-3f76ee04f851 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:11.636687 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0edda527-c668-4195-b705-3f76ee04f851 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:11.637119 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0edda527-c668-4195-b705-3f76ee04f851 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:11.637811 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0edda527-c668-4195-b705-3f76ee04f851 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:11.638257 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0edda527-c668-4195-b705-3f76ee04f851 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:11.638676 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0edda527-c668-4195-b705-3f76ee04f851 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:11.754875 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-99346a46-a924-4d19-85b2-5807daed38f6 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:11.755460 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-99346a46-a924-4d19-85b2-5807daed38f6 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 6eaaf253-16e7-4c62-a434-a4b8f5d7b536] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:11.761659 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-99346a46-a924-4d19-85b2-5807daed38f6 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:11.762145 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-99346a46-a924-4d19-85b2-5807daed38f6 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:11.762579 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-99346a46-a924-4d19-85b2-5807daed38f6 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:11.763272 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-99346a46-a924-4d19-85b2-5807daed38f6 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:11.763747 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-99346a46-a924-4d19-85b2-5807daed38f6 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:11.764176 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-99346a46-a924-4d19-85b2-5807daed38f6 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:11.784833 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-0edda527-c668-4195-b705-3f76ee04f851 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:11.792812 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-0edda527-c668-4195-b705-3f76ee04f851 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1321 microversion: 2.1 time: 0.172692 Oct 30 00:59:11.793354 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 159/329] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:11 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1321 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:11.893415 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-99346a46-a924-4d19-85b2-5807daed38f6 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:11.901461 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-99346a46-a924-4d19-85b2-5807daed38f6 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536" status: 200 len: 1313 microversion: 2.1 time: 0.150680 Oct 30 00:59:11.902000 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 171/330] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:11 2022] GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536 => generated 1313 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:12.820594 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-52054b1c-685a-4c60-9a3a-203bfbf3a4b6 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:12.821647 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-52054b1c-685a-4c60-9a3a-203bfbf3a4b6 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:12.833158 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-52054b1c-685a-4c60-9a3a-203bfbf3a4b6 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:12.833959 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-52054b1c-685a-4c60-9a3a-203bfbf3a4b6 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:12.834728 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-52054b1c-685a-4c60-9a3a-203bfbf3a4b6 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:12.836209 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-52054b1c-685a-4c60-9a3a-203bfbf3a4b6 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:12.837010 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-52054b1c-685a-4c60-9a3a-203bfbf3a4b6 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:12.837743 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-52054b1c-685a-4c60-9a3a-203bfbf3a4b6 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:12.932917 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-3f7c13d4-ccc3-450d-aede-9a8dec69bd9d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:12.933533 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-3f7c13d4-ccc3-450d-aede-9a8dec69bd9d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 6eaaf253-16e7-4c62-a434-a4b8f5d7b536] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:12.939884 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3f7c13d4-ccc3-450d-aede-9a8dec69bd9d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:12.940350 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3f7c13d4-ccc3-450d-aede-9a8dec69bd9d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:12.940788 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3f7c13d4-ccc3-450d-aede-9a8dec69bd9d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:12.941486 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3f7c13d4-ccc3-450d-aede-9a8dec69bd9d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:12.941939 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3f7c13d4-ccc3-450d-aede-9a8dec69bd9d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:12.942359 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3f7c13d4-ccc3-450d-aede-9a8dec69bd9d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:12.972953 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-52054b1c-685a-4c60-9a3a-203bfbf3a4b6 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:12.981187 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-52054b1c-685a-4c60-9a3a-203bfbf3a4b6 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1321 microversion: 2.1 time: 0.167264 Oct 30 00:59:12.981768 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 160/331] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:12 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1321 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:13.071285 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-3f7c13d4-ccc3-450d-aede-9a8dec69bd9d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:13.080028 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-3f7c13d4-ccc3-450d-aede-9a8dec69bd9d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536" status: 200 len: 1313 microversion: 2.1 time: 0.150927 Oct 30 00:59:13.080596 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 172/332] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:12 2022] GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536 => generated 1313 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:14.009779 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-fabd97f2-7ed2-4c8b-b0ef-2020598ff52c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:14.010791 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-fabd97f2-7ed2-4c8b-b0ef-2020598ff52c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:14.021572 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-fabd97f2-7ed2-4c8b-b0ef-2020598ff52c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:14.022374 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-fabd97f2-7ed2-4c8b-b0ef-2020598ff52c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:14.023207 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-fabd97f2-7ed2-4c8b-b0ef-2020598ff52c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:14.023983 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-fabd97f2-7ed2-4c8b-b0ef-2020598ff52c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:14.024403 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-fabd97f2-7ed2-4c8b-b0ef-2020598ff52c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:14.024822 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-fabd97f2-7ed2-4c8b-b0ef-2020598ff52c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:14.103393 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-cd00d2a7-4915-4283-876e-d9d9199dd47b tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:14.104014 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-cd00d2a7-4915-4283-876e-d9d9199dd47b tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 6eaaf253-16e7-4c62-a434-a4b8f5d7b536] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:14.110504 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cd00d2a7-4915-4283-876e-d9d9199dd47b tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:14.110992 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cd00d2a7-4915-4283-876e-d9d9199dd47b tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:14.111444 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cd00d2a7-4915-4283-876e-d9d9199dd47b tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:14.112141 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cd00d2a7-4915-4283-876e-d9d9199dd47b tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:14.112585 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cd00d2a7-4915-4283-876e-d9d9199dd47b tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:14.113008 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cd00d2a7-4915-4283-876e-d9d9199dd47b tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:14.167883 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-fabd97f2-7ed2-4c8b-b0ef-2020598ff52c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:14.175959 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-fabd97f2-7ed2-4c8b-b0ef-2020598ff52c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1321 microversion: 2.1 time: 0.172997 Oct 30 00:59:14.176500 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 161/333] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:14 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1321 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:14.248781 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-cd00d2a7-4915-4283-876e-d9d9199dd47b tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:14.256967 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-cd00d2a7-4915-4283-876e-d9d9199dd47b tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536" status: 200 len: 1313 microversion: 2.1 time: 0.157647 Oct 30 00:59:14.257513 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 173/334] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:14 2022] GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536 => generated 1313 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:15.200701 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-3c0c62d4-0281-438b-86c8-90c4aa2e4c18 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:15.201405 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-3c0c62d4-0281-438b-86c8-90c4aa2e4c18 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:15.209458 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3c0c62d4-0281-438b-86c8-90c4aa2e4c18 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:15.210024 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3c0c62d4-0281-438b-86c8-90c4aa2e4c18 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:15.210529 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3c0c62d4-0281-438b-86c8-90c4aa2e4c18 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:15.211335 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3c0c62d4-0281-438b-86c8-90c4aa2e4c18 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:15.211895 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3c0c62d4-0281-438b-86c8-90c4aa2e4c18 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:15.212635 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3c0c62d4-0281-438b-86c8-90c4aa2e4c18 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:15.280184 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-b37e5eb3-548e-4945-8169-54e1d0fa0d60 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:15.280899 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-b37e5eb3-548e-4945-8169-54e1d0fa0d60 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 6eaaf253-16e7-4c62-a434-a4b8f5d7b536] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:15.286945 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b37e5eb3-548e-4945-8169-54e1d0fa0d60 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:15.287426 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b37e5eb3-548e-4945-8169-54e1d0fa0d60 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:15.287877 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b37e5eb3-548e-4945-8169-54e1d0fa0d60 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:15.288571 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b37e5eb3-548e-4945-8169-54e1d0fa0d60 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:15.289146 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b37e5eb3-548e-4945-8169-54e1d0fa0d60 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:15.289569 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b37e5eb3-548e-4945-8169-54e1d0fa0d60 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:15.378022 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-3c0c62d4-0281-438b-86c8-90c4aa2e4c18 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:15.385830 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-3c0c62d4-0281-438b-86c8-90c4aa2e4c18 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1321 microversion: 2.1 time: 0.190416 Oct 30 00:59:15.386598 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 162/335] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:15 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1321 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:15.438055 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-b37e5eb3-548e-4945-8169-54e1d0fa0d60 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:15.446645 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-b37e5eb3-548e-4945-8169-54e1d0fa0d60 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536" status: 200 len: 1313 microversion: 2.1 time: 0.170380 Oct 30 00:59:15.447224 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 174/336] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:15 2022] GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536 => generated 1313 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:16.412773 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-b0020abf-f687-4557-b15a-55d15f67e1e2 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:16.413776 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-b0020abf-f687-4557-b15a-55d15f67e1e2 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:16.425152 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b0020abf-f687-4557-b15a-55d15f67e1e2 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:16.425986 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b0020abf-f687-4557-b15a-55d15f67e1e2 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:16.426667 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b0020abf-f687-4557-b15a-55d15f67e1e2 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:16.427801 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b0020abf-f687-4557-b15a-55d15f67e1e2 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:16.428584 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b0020abf-f687-4557-b15a-55d15f67e1e2 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:16.429277 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b0020abf-f687-4557-b15a-55d15f67e1e2 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:16.470529 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-153116fc-a0ff-4670-b9f1-90ea6fb1dbd3 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:16.471166 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-153116fc-a0ff-4670-b9f1-90ea6fb1dbd3 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 6eaaf253-16e7-4c62-a434-a4b8f5d7b536] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:16.483032 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-153116fc-a0ff-4670-b9f1-90ea6fb1dbd3 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:16.483533 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-153116fc-a0ff-4670-b9f1-90ea6fb1dbd3 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:16.483999 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-153116fc-a0ff-4670-b9f1-90ea6fb1dbd3 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:16.484750 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-153116fc-a0ff-4670-b9f1-90ea6fb1dbd3 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:16.485150 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-153116fc-a0ff-4670-b9f1-90ea6fb1dbd3 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:16.485621 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-153116fc-a0ff-4670-b9f1-90ea6fb1dbd3 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:16.674188 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-b0020abf-f687-4557-b15a-55d15f67e1e2 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:16.685819 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-153116fc-a0ff-4670-b9f1-90ea6fb1dbd3 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:16.689164 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-b0020abf-f687-4557-b15a-55d15f67e1e2 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1321 microversion: 2.1 time: 0.282629 Oct 30 00:59:16.690044 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 163/337] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:16 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1321 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:16.712145 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-153116fc-a0ff-4670-b9f1-90ea6fb1dbd3 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536" status: 200 len: 1313 microversion: 2.1 time: 0.241416 Oct 30 00:59:16.712775 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 175/338] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:16 2022] GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536 => generated 1313 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:17.712017 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-4e0e6b77-2835-48be-a635-1f10da5997bc tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:17.712792 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-4e0e6b77-2835-48be-a635-1f10da5997bc tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:17.719768 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4e0e6b77-2835-48be-a635-1f10da5997bc tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:17.720258 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4e0e6b77-2835-48be-a635-1f10da5997bc tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:17.720700 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4e0e6b77-2835-48be-a635-1f10da5997bc tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:17.721461 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4e0e6b77-2835-48be-a635-1f10da5997bc tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:17.721855 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4e0e6b77-2835-48be-a635-1f10da5997bc tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:17.722265 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4e0e6b77-2835-48be-a635-1f10da5997bc tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:17.735365 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-81b002d0-3003-4b96-90c7-ad6520fe82ca tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:17.736055 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-81b002d0-3003-4b96-90c7-ad6520fe82ca tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 6eaaf253-16e7-4c62-a434-a4b8f5d7b536] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:17.742458 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-81b002d0-3003-4b96-90c7-ad6520fe82ca tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:17.743153 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-81b002d0-3003-4b96-90c7-ad6520fe82ca tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:17.743643 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-81b002d0-3003-4b96-90c7-ad6520fe82ca tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:17.744631 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-81b002d0-3003-4b96-90c7-ad6520fe82ca tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:17.745299 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-81b002d0-3003-4b96-90c7-ad6520fe82ca tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:17.745727 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-81b002d0-3003-4b96-90c7-ad6520fe82ca tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:17.922974 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-4e0e6b77-2835-48be-a635-1f10da5997bc tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:17.932362 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-4e0e6b77-2835-48be-a635-1f10da5997bc tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1321 microversion: 2.1 time: 0.223811 Oct 30 00:59:17.933770 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-81b002d0-3003-4b96-90c7-ad6520fe82ca tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:17.933941 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 164/339] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:17 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1321 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:17.944618 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-81b002d0-3003-4b96-90c7-ad6520fe82ca tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536" status: 200 len: 1313 microversion: 2.1 time: 0.213110 Oct 30 00:59:17.951677 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 176/340] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:17 2022] GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536 => generated 1313 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 00:59:18.957731 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-dda0717b-34d6-471d-b920-cda63aa4b43e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:18.958438 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-dda0717b-34d6-471d-b920-cda63aa4b43e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:18.965972 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dda0717b-34d6-471d-b920-cda63aa4b43e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:18.966538 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dda0717b-34d6-471d-b920-cda63aa4b43e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:18.967094 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dda0717b-34d6-471d-b920-cda63aa4b43e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:18.967849 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dda0717b-34d6-471d-b920-cda63aa4b43e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:18.968302 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dda0717b-34d6-471d-b920-cda63aa4b43e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:18.968730 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dda0717b-34d6-471d-b920-cda63aa4b43e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:18.972374 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-20a04708-d64b-4d5b-a8d3-369c7ea0f628 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:18.972987 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-20a04708-d64b-4d5b-a8d3-369c7ea0f628 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 6eaaf253-16e7-4c62-a434-a4b8f5d7b536] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:18.979026 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-20a04708-d64b-4d5b-a8d3-369c7ea0f628 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:18.979513 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-20a04708-d64b-4d5b-a8d3-369c7ea0f628 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:18.980044 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-20a04708-d64b-4d5b-a8d3-369c7ea0f628 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:18.980792 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-20a04708-d64b-4d5b-a8d3-369c7ea0f628 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:18.981664 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-20a04708-d64b-4d5b-a8d3-369c7ea0f628 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:18.982209 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-20a04708-d64b-4d5b-a8d3-369c7ea0f628 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:19.166087 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-dda0717b-34d6-471d-b920-cda63aa4b43e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:19.176588 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-20a04708-d64b-4d5b-a8d3-369c7ea0f628 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:19.182391 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-dda0717b-34d6-471d-b920-cda63aa4b43e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1321 microversion: 2.1 time: 0.228525 Oct 30 00:59:19.195666 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 165/341] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:18 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1321 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:19.196083 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-20a04708-d64b-4d5b-a8d3-369c7ea0f628 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536" status: 200 len: 1313 microversion: 2.1 time: 0.217791 Oct 30 00:59:19.196083 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 177/342] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:18 2022] GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536 => generated 1313 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 00:59:19.223431 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [req-65fc06de-9926-4c67-9459-3a02a5fa8876 req-c0b9e652-b346-4185-9906-d4858eb69940 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6eaaf253-16e7-4c62-a434-a4b8f5d7b536", "name": "network-vif-plugged", "status": "completed", "tag": "789b3d8f-15d1-4b53-9724-f962176bd31c"}]} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:59:19.233942 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-65fc06de-9926-4c67-9459-3a02a5fa8876 req-c0b9e652-b346-4185-9906-d4858eb69940 service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:19.234445 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-65fc06de-9926-4c67-9459-3a02a5fa8876 req-c0b9e652-b346-4185-9906-d4858eb69940 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:19.234844 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-65fc06de-9926-4c67-9459-3a02a5fa8876 req-c0b9e652-b346-4185-9906-d4858eb69940 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:19.246804 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.compute.server_external_events [req-65fc06de-9926-4c67-9459-3a02a5fa8876 req-c0b9e652-b346-4185-9906-d4858eb69940 service nova] Creating event network-vif-plugged:789b3d8f-15d1-4b53-9724-f962176bd31c for instance 6eaaf253-16e7-4c62-a434-a4b8f5d7b536 on hv-cinder-maste Oct 30 00:59:19.251982 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [req-65fc06de-9926-4c67-9459-3a02a5fa8876 req-c0b9e652-b346-4185-9906-d4858eb69940 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032487 Oct 30 00:59:19.252580 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 166/343] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 00:59:19 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:59:20.208656 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-5eb41731-3bc0-45ab-a604-be1e6ad83614 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:20.209753 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-5eb41731-3bc0-45ab-a604-be1e6ad83614 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:20.214190 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-eef663e2-975c-4b16-99aa-7194726bdb1b tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:20.216052 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-eef663e2-975c-4b16-99aa-7194726bdb1b tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 6eaaf253-16e7-4c62-a434-a4b8f5d7b536] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:20.223549 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-eef663e2-975c-4b16-99aa-7194726bdb1b tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:20.224082 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-eef663e2-975c-4b16-99aa-7194726bdb1b tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:20.224696 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-eef663e2-975c-4b16-99aa-7194726bdb1b tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:20.225004 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5eb41731-3bc0-45ab-a604-be1e6ad83614 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:20.225778 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5eb41731-3bc0-45ab-a604-be1e6ad83614 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:20.225926 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-eef663e2-975c-4b16-99aa-7194726bdb1b tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:20.226270 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-eef663e2-975c-4b16-99aa-7194726bdb1b tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:20.226902 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-eef663e2-975c-4b16-99aa-7194726bdb1b tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:20.229060 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5eb41731-3bc0-45ab-a604-be1e6ad83614 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:20.229871 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5eb41731-3bc0-45ab-a604-be1e6ad83614 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:20.230402 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5eb41731-3bc0-45ab-a604-be1e6ad83614 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:20.231189 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5eb41731-3bc0-45ab-a604-be1e6ad83614 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:20.440719 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-eef663e2-975c-4b16-99aa-7194726bdb1b tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:20.448655 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-eef663e2-975c-4b16-99aa-7194726bdb1b tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536" status: 200 len: 1313 microversion: 2.1 time: 0.242894 Oct 30 00:59:20.450042 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 167/344] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:20 2022] GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536 => generated 1313 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 00:59:20.453910 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-5eb41731-3bc0-45ab-a604-be1e6ad83614 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:20.462127 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-5eb41731-3bc0-45ab-a604-be1e6ad83614 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1516 microversion: 2.1 time: 0.260831 Oct 30 00:59:20.462865 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 178/345] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:20 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1516 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:20.485539 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-51a05aa0-7179-4379-9d02-6889d8c4f1a1 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:59:20.487765 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-51a05aa0-7179-4379-9d02-6889d8c4f1a1 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:20.493876 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-51a05aa0-7179-4379-9d02-6889d8c4f1a1 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:20.494352 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-51a05aa0-7179-4379-9d02-6889d8c4f1a1 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:20.494781 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-51a05aa0-7179-4379-9d02-6889d8c4f1a1 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:20.495474 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-51a05aa0-7179-4379-9d02-6889d8c4f1a1 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:20.496304 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-51a05aa0-7179-4379-9d02-6889d8c4f1a1 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:20.497725 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-51a05aa0-7179-4379-9d02-6889d8c4f1a1 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:20.536793 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.objects.instance [None req-51a05aa0-7179-4379-9d02-6889d8c4f1a1 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lazy-loading 'flavor' on Instance uuid 60b2f6e0-a160-4d3f-b036-38597b60da68 {{(pid=118962) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 30 00:59:20.670358 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=2 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:59:20.719556 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-51a05aa0-7179-4379-9d02-6889d8c4f1a1 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "POST /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68/action" status: 200 len: 32 microversion: 2.1 time: 0.236749 Oct 30 00:59:20.720545 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 168/346] 10.222.0.16 () {60 vars in 1278 bytes} [Sun Oct 30 00:59:20 2022] POST /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68/action => generated 32 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 30 00:59:20.738831 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-a12289ec-02fb-425f-9a45-c945b86b78e5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:20.742860 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-a12289ec-02fb-425f-9a45-c945b86b78e5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:20.759527 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a12289ec-02fb-425f-9a45-c945b86b78e5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:20.763193 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a12289ec-02fb-425f-9a45-c945b86b78e5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:20.763672 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a12289ec-02fb-425f-9a45-c945b86b78e5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:20.764436 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a12289ec-02fb-425f-9a45-c945b86b78e5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:20.764896 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a12289ec-02fb-425f-9a45-c945b86b78e5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:20.765317 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a12289ec-02fb-425f-9a45-c945b86b78e5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:20.911257 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-a12289ec-02fb-425f-9a45-c945b86b78e5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:20.919600 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-a12289ec-02fb-425f-9a45-c945b86b78e5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1522 microversion: 2.1 time: 0.192366 Oct 30 00:59:20.920289 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 179/347] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:20 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1522 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:21.473921 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-5d4af040-2e9d-4950-8311-6a272d73029a tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:21.474627 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-5d4af040-2e9d-4950-8311-6a272d73029a tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 6eaaf253-16e7-4c62-a434-a4b8f5d7b536] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:21.476656 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_api:SELECT=54 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:59:21.483000 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5d4af040-2e9d-4950-8311-6a272d73029a tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:21.483479 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5d4af040-2e9d-4950-8311-6a272d73029a tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:21.483975 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5d4af040-2e9d-4950-8311-6a272d73029a tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:21.484661 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5d4af040-2e9d-4950-8311-6a272d73029a tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:21.485106 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5d4af040-2e9d-4950-8311-6a272d73029a tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:21.485522 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5d4af040-2e9d-4950-8311-6a272d73029a tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:21.622562 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-5d4af040-2e9d-4950-8311-6a272d73029a tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:21.630656 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-5d4af040-2e9d-4950-8311-6a272d73029a tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536" status: 200 len: 1313 microversion: 2.1 time: 0.161686 Oct 30 00:59:21.631217 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 169/348] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:21 2022] GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536 => generated 1313 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:21.942260 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-28d74187-b7bd-415b-8004-de5bf0777ff6 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:21.942943 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-28d74187-b7bd-415b-8004-de5bf0777ff6 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:21.944952 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_api:SELECT=54 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:59:21.950408 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-28d74187-b7bd-415b-8004-de5bf0777ff6 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:21.950897 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-28d74187-b7bd-415b-8004-de5bf0777ff6 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:21.951331 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-28d74187-b7bd-415b-8004-de5bf0777ff6 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:21.952065 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-28d74187-b7bd-415b-8004-de5bf0777ff6 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:21.952585 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-28d74187-b7bd-415b-8004-de5bf0777ff6 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:21.953118 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-28d74187-b7bd-415b-8004-de5bf0777ff6 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:22.089529 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-28d74187-b7bd-415b-8004-de5bf0777ff6 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:22.097679 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-28d74187-b7bd-415b-8004-de5bf0777ff6 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1522 microversion: 2.1 time: 0.159612 Oct 30 00:59:22.098237 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 180/349] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:21 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1522 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:22.654424 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-32994f48-7b7e-4a5d-a793-c50fc17b01a8 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:22.655106 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-32994f48-7b7e-4a5d-a793-c50fc17b01a8 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 6eaaf253-16e7-4c62-a434-a4b8f5d7b536] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:22.662111 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-32994f48-7b7e-4a5d-a793-c50fc17b01a8 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:22.662592 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-32994f48-7b7e-4a5d-a793-c50fc17b01a8 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:22.663014 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-32994f48-7b7e-4a5d-a793-c50fc17b01a8 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:22.663736 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-32994f48-7b7e-4a5d-a793-c50fc17b01a8 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:22.664194 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-32994f48-7b7e-4a5d-a793-c50fc17b01a8 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:22.664607 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-32994f48-7b7e-4a5d-a793-c50fc17b01a8 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:22.816850 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-32994f48-7b7e-4a5d-a793-c50fc17b01a8 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:22.826130 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-32994f48-7b7e-4a5d-a793-c50fc17b01a8 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536" status: 200 len: 1313 microversion: 2.1 time: 0.175530 Oct 30 00:59:22.826725 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 170/350] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:22 2022] GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536 => generated 1313 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:23.124817 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-cbd7ae8e-e6ad-49ed-91ba-93d5e42dfe22 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:23.125489 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-cbd7ae8e-e6ad-49ed-91ba-93d5e42dfe22 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:23.132945 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cbd7ae8e-e6ad-49ed-91ba-93d5e42dfe22 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:23.133498 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cbd7ae8e-e6ad-49ed-91ba-93d5e42dfe22 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:23.133981 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cbd7ae8e-e6ad-49ed-91ba-93d5e42dfe22 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:23.134746 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cbd7ae8e-e6ad-49ed-91ba-93d5e42dfe22 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:23.135255 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cbd7ae8e-e6ad-49ed-91ba-93d5e42dfe22 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:23.135741 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cbd7ae8e-e6ad-49ed-91ba-93d5e42dfe22 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:23.141340 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell1:SELECT=100 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:59:23.277377 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-cbd7ae8e-e6ad-49ed-91ba-93d5e42dfe22 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:23.286182 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-cbd7ae8e-e6ad-49ed-91ba-93d5e42dfe22 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1522 microversion: 2.1 time: 0.167390 Oct 30 00:59:23.286776 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 181/351] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:23 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1522 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:23.708327 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-f3ad3a11-0ad9-4e2c-a52d-a90c9ca41388 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesNegativeTest-instance-1644054862", "imageRef": "4c16f4f6-84cb-4463-b82c-b7c65f3826f0", "flavorRef": "452"}} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:59:23.762515 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.network.neutron [None req-f3ad3a11-0ad9-4e2c-a52d-a90c9ca41388 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] validate_networks() for None {{(pid=118962) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 30 00:59:23.848777 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-0b8cdb85-9cf0-4ce9-9da6-fcb17d34a5e7 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:23.849384 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-0b8cdb85-9cf0-4ce9-9da6-fcb17d34a5e7 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 6eaaf253-16e7-4c62-a434-a4b8f5d7b536] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:23.855949 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0b8cdb85-9cf0-4ce9-9da6-fcb17d34a5e7 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:23.856448 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0b8cdb85-9cf0-4ce9-9da6-fcb17d34a5e7 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:23.856874 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0b8cdb85-9cf0-4ce9-9da6-fcb17d34a5e7 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:23.857575 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0b8cdb85-9cf0-4ce9-9da6-fcb17d34a5e7 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:23.858019 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0b8cdb85-9cf0-4ce9-9da6-fcb17d34a5e7 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:23.858434 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0b8cdb85-9cf0-4ce9-9da6-fcb17d34a5e7 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:24.291587 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-0b8cdb85-9cf0-4ce9-9da6-fcb17d34a5e7 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:24.299958 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-0b8cdb85-9cf0-4ce9-9da6-fcb17d34a5e7 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536" status: 200 len: 1496 microversion: 2.1 time: 0.455011 Oct 30 00:59:24.300527 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 182/352] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:23 2022] GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536 => generated 1496 bytes in 456 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:24.310278 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-85e07ad2-4b4c-465f-b374-4e9b531dc6f3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:24.311242 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-85e07ad2-4b4c-465f-b374-4e9b531dc6f3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:24.317691 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-85e07ad2-4b4c-465f-b374-4e9b531dc6f3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:24.318357 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-85e07ad2-4b4c-465f-b374-4e9b531dc6f3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:24.318956 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-85e07ad2-4b4c-465f-b374-4e9b531dc6f3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:24.320427 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-85e07ad2-4b4c-465f-b374-4e9b531dc6f3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:24.320859 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-85e07ad2-4b4c-465f-b374-4e9b531dc6f3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:24.321465 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-85e07ad2-4b4c-465f-b374-4e9b531dc6f3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:24.335739 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.virt.hardware [None req-f3ad3a11-0ad9-4e2c-a52d-a90c9ca41388 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Flavor limits 0:0:0 {{(pid=118962) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 30 00:59:24.336198 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.virt.hardware [None req-f3ad3a11-0ad9-4e2c-a52d-a90c9ca41388 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Image limits 0:0:0 {{(pid=118962) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 30 00:59:24.337052 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.virt.hardware [None req-f3ad3a11-0ad9-4e2c-a52d-a90c9ca41388 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Flavor pref 0:0:0 {{(pid=118962) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 30 00:59:24.337456 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.virt.hardware [None req-f3ad3a11-0ad9-4e2c-a52d-a90c9ca41388 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Image pref 0:0:0 {{(pid=118962) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 30 00:59:24.337854 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.virt.hardware [None req-f3ad3a11-0ad9-4e2c-a52d-a90c9ca41388 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118962) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 30 00:59:24.360414 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f3ad3a11-0ad9-4e2c-a52d-a90c9ca41388 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:24.361003 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f3ad3a11-0ad9-4e2c-a52d-a90c9ca41388 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:24.361528 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f3ad3a11-0ad9-4e2c-a52d-a90c9ca41388 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:24.362363 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f3ad3a11-0ad9-4e2c-a52d-a90c9ca41388 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:24.363062 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f3ad3a11-0ad9-4e2c-a52d-a90c9ca41388 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:24.363810 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f3ad3a11-0ad9-4e2c-a52d-a90c9ca41388 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:24.386173 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.quota [None req-f3ad3a11-0ad9-4e2c-a52d-a90c9ca41388 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Getting quotas for project 27fc853acd7b4ef3803aec20a793bd06. Resources: {'ram', 'instances', 'cores'} {{(pid=118962) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 30 00:59:24.390197 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.quota [None req-f3ad3a11-0ad9-4e2c-a52d-a90c9ca41388 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Getting quotas for user bc8294d49c654c8f9d8f7539b359df85 and project 27fc853acd7b4ef3803aec20a793bd06. Resources: {'ram', 'instances', 'cores'} {{(pid=118962) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 30 00:59:24.403394 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-f3ad3a11-0ad9-4e2c-a52d-a90c9ca41388 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Going to run 1 instances... {{(pid=118962) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 30 00:59:24.405881 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-f3ad3a11-0ad9-4e2c-a52d-a90c9ca41388 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] [instance: 0dc14c92-6364-4981-81c3-1c00e4f8187b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4c16f4f6-84cb-4463-b82c-b7c65f3826f0',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118962) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 30 00:59:24.452580 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-f3ad3a11-0ad9-4e2c-a52d-a90c9ca41388 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.747983 Oct 30 00:59:24.453510 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 171/353] 10.222.0.16 () {60 vars in 1147 bytes} [Sun Oct 30 00:59:23 2022] POST /compute/v2.1/servers => generated 380 bytes in 749 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 30 00:59:24.468896 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-972f9725-edbc-4fdc-82cb-c19e1e5f3c65 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:24.469696 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-972f9725-edbc-4fdc-82cb-c19e1e5f3c65 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] [instance: 0dc14c92-6364-4981-81c3-1c00e4f8187b] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:24.473400 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-85e07ad2-4b4c-465f-b374-4e9b531dc6f3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:24.482492 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-85e07ad2-4b4c-465f-b374-4e9b531dc6f3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1522 microversion: 2.1 time: 0.176266 Oct 30 00:59:24.483060 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 183/354] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:24 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1522 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:24.525216 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-972f9725-edbc-4fdc-82cb-c19e1e5f3c65 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc8294d49c654c8f9d8f7539b359df85', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27fc853acd7b4ef3803aec20a793bd06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:24.533790 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-972f9725-edbc-4fdc-82cb-c19e1e5f3c65 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] 10.222.0.16 "GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b" status: 200 len: 1215 microversion: 2.1 time: 0.073220 Oct 30 00:59:24.534384 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 172/355] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:24 2022] GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b => generated 1215 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 00:59:24.881693 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-995060a3-db1e-4c9b-8806-afab3cc6a268 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:24.882249 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-995060a3-db1e-4c9b-8806-afab3cc6a268 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 6eaaf253-16e7-4c62-a434-a4b8f5d7b536] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:24.888849 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-995060a3-db1e-4c9b-8806-afab3cc6a268 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:24.889337 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-995060a3-db1e-4c9b-8806-afab3cc6a268 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:24.889817 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-995060a3-db1e-4c9b-8806-afab3cc6a268 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:24.890543 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-995060a3-db1e-4c9b-8806-afab3cc6a268 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:24.891059 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-995060a3-db1e-4c9b-8806-afab3cc6a268 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:24.891550 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-995060a3-db1e-4c9b-8806-afab3cc6a268 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:24.918737 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-995060a3-db1e-4c9b-8806-afab3cc6a268 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 6eaaf253-16e7-4c62-a434-a4b8f5d7b536] Going to try to terminate instance {{(pid=118961) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 30 00:59:25.005834 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-995060a3-db1e-4c9b-8806-afab3cc6a268 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "DELETE /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536" status: 204 len: 0 microversion: 2.1 time: 0.128759 Oct 30 00:59:25.006968 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 184/356] 10.222.0.16 () {58 vars in 1239 bytes} [Sun Oct 30 00:59:24 2022] DELETE /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 30 00:59:25.015298 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-dbca39bb-a780-4ab4-9071-66696c48f5d4 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:25.015934 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-dbca39bb-a780-4ab4-9071-66696c48f5d4 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 6eaaf253-16e7-4c62-a434-a4b8f5d7b536] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:25.022371 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dbca39bb-a780-4ab4-9071-66696c48f5d4 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:25.022842 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dbca39bb-a780-4ab4-9071-66696c48f5d4 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:25.023265 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dbca39bb-a780-4ab4-9071-66696c48f5d4 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:25.023973 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dbca39bb-a780-4ab4-9071-66696c48f5d4 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:25.024415 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dbca39bb-a780-4ab4-9071-66696c48f5d4 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:25.024959 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dbca39bb-a780-4ab4-9071-66696c48f5d4 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:25.173866 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-dbca39bb-a780-4ab4-9071-66696c48f5d4 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:25.182155 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-dbca39bb-a780-4ab4-9071-66696c48f5d4 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536" status: 200 len: 1502 microversion: 2.1 time: 0.170722 Oct 30 00:59:25.182722 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 173/357] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:25 2022] GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536 => generated 1502 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:25.510643 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-14ac8c6e-74b6-4d0a-89d2-9cc392742d62 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:25.511776 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-14ac8c6e-74b6-4d0a-89d2-9cc392742d62 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:25.518869 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-14ac8c6e-74b6-4d0a-89d2-9cc392742d62 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:25.519361 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-14ac8c6e-74b6-4d0a-89d2-9cc392742d62 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:25.519815 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-14ac8c6e-74b6-4d0a-89d2-9cc392742d62 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:25.520511 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-14ac8c6e-74b6-4d0a-89d2-9cc392742d62 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:25.520960 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-14ac8c6e-74b6-4d0a-89d2-9cc392742d62 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:25.521375 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-14ac8c6e-74b6-4d0a-89d2-9cc392742d62 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:25.557260 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-88ae49ed-9b48-4a75-9394-c60dd1242aad tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:25.557976 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-88ae49ed-9b48-4a75-9394-c60dd1242aad tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] [instance: 0dc14c92-6364-4981-81c3-1c00e4f8187b] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:25.564449 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-88ae49ed-9b48-4a75-9394-c60dd1242aad tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:25.564940 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-88ae49ed-9b48-4a75-9394-c60dd1242aad tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:25.565373 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-88ae49ed-9b48-4a75-9394-c60dd1242aad tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:25.566075 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-88ae49ed-9b48-4a75-9394-c60dd1242aad tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:25.566528 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-88ae49ed-9b48-4a75-9394-c60dd1242aad tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:25.566946 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-88ae49ed-9b48-4a75-9394-c60dd1242aad tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:25.643450 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-88ae49ed-9b48-4a75-9394-c60dd1242aad tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc8294d49c654c8f9d8f7539b359df85', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27fc853acd7b4ef3803aec20a793bd06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:25.651706 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-88ae49ed-9b48-4a75-9394-c60dd1242aad tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] 10.222.0.16 "GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b" status: 200 len: 1211 microversion: 2.1 time: 0.098282 Oct 30 00:59:25.652613 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 174/358] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:25 2022] GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b => generated 1211 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:25.679913 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-14ac8c6e-74b6-4d0a-89d2-9cc392742d62 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:25.688309 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-14ac8c6e-74b6-4d0a-89d2-9cc392742d62 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1522 microversion: 2.1 time: 0.184744 Oct 30 00:59:25.688949 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 185/359] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:25 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1522 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:26.210518 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-e2ea438b-3e7c-4848-ac7d-d5081db6f291 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:26.211530 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-e2ea438b-3e7c-4848-ac7d-d5081db6f291 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 6eaaf253-16e7-4c62-a434-a4b8f5d7b536] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:26.222588 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e2ea438b-3e7c-4848-ac7d-d5081db6f291 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:26.223406 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e2ea438b-3e7c-4848-ac7d-d5081db6f291 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:26.224178 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e2ea438b-3e7c-4848-ac7d-d5081db6f291 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:26.225308 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e2ea438b-3e7c-4848-ac7d-d5081db6f291 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:26.226082 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e2ea438b-3e7c-4848-ac7d-d5081db6f291 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:26.226766 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e2ea438b-3e7c-4848-ac7d-d5081db6f291 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:26.370739 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-e2ea438b-3e7c-4848-ac7d-d5081db6f291 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:26.379925 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-e2ea438b-3e7c-4848-ac7d-d5081db6f291 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536" status: 200 len: 1502 microversion: 2.1 time: 0.175911 Oct 30 00:59:26.380762 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 175/360] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:26 2022] GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536 => generated 1502 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:26.678941 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-f70b4f61-f055-4681-a82c-d5974676374d tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:26.679839 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-f70b4f61-f055-4681-a82c-d5974676374d tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] [instance: 0dc14c92-6364-4981-81c3-1c00e4f8187b] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:26.688814 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f70b4f61-f055-4681-a82c-d5974676374d tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:26.689299 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f70b4f61-f055-4681-a82c-d5974676374d tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:26.689729 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f70b4f61-f055-4681-a82c-d5974676374d tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:26.690440 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f70b4f61-f055-4681-a82c-d5974676374d tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:26.691240 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f70b4f61-f055-4681-a82c-d5974676374d tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:26.691240 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f70b4f61-f055-4681-a82c-d5974676374d tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:26.711168 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-a1777ce9-1f79-4b16-b7c8-e55a51a5e755 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:26.711963 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-a1777ce9-1f79-4b16-b7c8-e55a51a5e755 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:26.718721 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a1777ce9-1f79-4b16-b7c8-e55a51a5e755 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:26.719240 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a1777ce9-1f79-4b16-b7c8-e55a51a5e755 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:26.719788 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a1777ce9-1f79-4b16-b7c8-e55a51a5e755 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:26.720616 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a1777ce9-1f79-4b16-b7c8-e55a51a5e755 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:26.721143 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a1777ce9-1f79-4b16-b7c8-e55a51a5e755 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:26.721640 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a1777ce9-1f79-4b16-b7c8-e55a51a5e755 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:26.757940 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-f70b4f61-f055-4681-a82c-d5974676374d tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc8294d49c654c8f9d8f7539b359df85', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27fc853acd7b4ef3803aec20a793bd06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:26.766620 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-f70b4f61-f055-4681-a82c-d5974676374d tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] 10.222.0.16 "GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b" status: 200 len: 1267 microversion: 2.1 time: 0.093949 Oct 30 00:59:26.767392 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 186/361] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:26 2022] GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b => generated 1267 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:26.861384 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-a1777ce9-1f79-4b16-b7c8-e55a51a5e755 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:26.869691 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-a1777ce9-1f79-4b16-b7c8-e55a51a5e755 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1522 microversion: 2.1 time: 0.162792 Oct 30 00:59:26.870279 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 176/362] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:26 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1522 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 00:59:27.404660 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-d23071a3-acfd-4a0e-89c3-a7dfa3f3503f tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:27.405297 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-d23071a3-acfd-4a0e-89c3-a7dfa3f3503f tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 6eaaf253-16e7-4c62-a434-a4b8f5d7b536] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:27.411874 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d23071a3-acfd-4a0e-89c3-a7dfa3f3503f tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:27.412311 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d23071a3-acfd-4a0e-89c3-a7dfa3f3503f tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:27.412754 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d23071a3-acfd-4a0e-89c3-a7dfa3f3503f tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:27.413422 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d23071a3-acfd-4a0e-89c3-a7dfa3f3503f tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:27.413866 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d23071a3-acfd-4a0e-89c3-a7dfa3f3503f tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:27.414276 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d23071a3-acfd-4a0e-89c3-a7dfa3f3503f tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:27.549550 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-d23071a3-acfd-4a0e-89c3-a7dfa3f3503f tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:27.557760 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-d23071a3-acfd-4a0e-89c3-a7dfa3f3503f tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536" status: 200 len: 1502 microversion: 2.1 time: 0.157036 Oct 30 00:59:27.558602 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 187/363] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:27 2022] GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536 => generated 1502 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:27.790399 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-05d934cd-de64-4c87-91c6-497d69c29380 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:27.791934 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-05d934cd-de64-4c87-91c6-497d69c29380 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] [instance: 0dc14c92-6364-4981-81c3-1c00e4f8187b] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:27.799697 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-05d934cd-de64-4c87-91c6-497d69c29380 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:27.800407 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-05d934cd-de64-4c87-91c6-497d69c29380 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:27.801002 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-05d934cd-de64-4c87-91c6-497d69c29380 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:27.802148 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-05d934cd-de64-4c87-91c6-497d69c29380 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:27.802892 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-05d934cd-de64-4c87-91c6-497d69c29380 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:27.803457 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-05d934cd-de64-4c87-91c6-497d69c29380 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:27.874175 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-05d934cd-de64-4c87-91c6-497d69c29380 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc8294d49c654c8f9d8f7539b359df85', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27fc853acd7b4ef3803aec20a793bd06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:27.882254 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-05d934cd-de64-4c87-91c6-497d69c29380 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] 10.222.0.16 "GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b" status: 200 len: 1285 microversion: 2.1 time: 0.095715 Oct 30 00:59:27.882791 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 177/364] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:27 2022] GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b => generated 1285 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:27.893106 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-1e5e755f-c095-4707-998f-903482bc06e9 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:27.893650 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-1e5e755f-c095-4707-998f-903482bc06e9 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:27.900947 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1e5e755f-c095-4707-998f-903482bc06e9 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:27.901434 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1e5e755f-c095-4707-998f-903482bc06e9 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:27.901866 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1e5e755f-c095-4707-998f-903482bc06e9 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:27.902603 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1e5e755f-c095-4707-998f-903482bc06e9 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:27.903055 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1e5e755f-c095-4707-998f-903482bc06e9 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:27.903477 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1e5e755f-c095-4707-998f-903482bc06e9 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:28.033736 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-1e5e755f-c095-4707-998f-903482bc06e9 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:28.042080 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-1e5e755f-c095-4707-998f-903482bc06e9 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1522 microversion: 2.1 time: 0.152998 Oct 30 00:59:28.042654 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 188/365] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:27 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1522 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:28.565874 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [req-4e3ac747-e783-40e9-ab3a-1899d55675dd req-e7c351f1-9b60-4893-96df-2506568bd0bd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6eaaf253-16e7-4c62-a434-a4b8f5d7b536", "name": "network-vif-deleted", "tag": "789b3d8f-15d1-4b53-9724-f962176bd31c"}]} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:59:28.581288 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-4e3ac747-e783-40e9-ab3a-1899d55675dd req-e7c351f1-9b60-4893-96df-2506568bd0bd service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:28.581691 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-4e3ac747-e783-40e9-ab3a-1899d55675dd req-e7c351f1-9b60-4893-96df-2506568bd0bd service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:28.582186 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-4e3ac747-e783-40e9-ab3a-1899d55675dd req-e7c351f1-9b60-4893-96df-2506568bd0bd service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:28.582684 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-0eea188e-59e8-45c7-ae1f-929597fb00a3 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:28.583294 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-0eea188e-59e8-45c7-ae1f-929597fb00a3 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 6eaaf253-16e7-4c62-a434-a4b8f5d7b536] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:28.589920 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0eea188e-59e8-45c7-ae1f-929597fb00a3 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:28.590393 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0eea188e-59e8-45c7-ae1f-929597fb00a3 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:28.590842 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0eea188e-59e8-45c7-ae1f-929597fb00a3 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:28.591557 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0eea188e-59e8-45c7-ae1f-929597fb00a3 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:28.592092 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0eea188e-59e8-45c7-ae1f-929597fb00a3 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:28.592513 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0eea188e-59e8-45c7-ae1f-929597fb00a3 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:28.605265 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.compute.server_external_events [req-4e3ac747-e783-40e9-ab3a-1899d55675dd req-e7c351f1-9b60-4893-96df-2506568bd0bd service nova] Creating event network-vif-deleted:789b3d8f-15d1-4b53-9724-f962176bd31c for instance 6eaaf253-16e7-4c62-a434-a4b8f5d7b536 on hv-cinder-maste Oct 30 00:59:28.613168 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [req-4e3ac747-e783-40e9-ab3a-1899d55675dd req-e7c351f1-9b60-4893-96df-2506568bd0bd service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.050980 Oct 30 00:59:28.614655 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 178/366] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 00:59:28 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 30 00:59:28.673538 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-0eea188e-59e8-45c7-ae1f-929597fb00a3 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:28.682363 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-0eea188e-59e8-45c7-ae1f-929597fb00a3 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536" status: 200 len: 1460 microversion: 2.1 time: 0.103487 Oct 30 00:59:28.682952 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 189/367] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:28 2022] GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536 => generated 1460 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 00:59:28.891318 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-45201128-e5f2-4275-8217-dc6a541551b2 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "cbb8a932-08ad-4afc-a8b0-b4f53b511ecc", "device": "/dev/sdb"}} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:59:28.894438 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-45201128-e5f2-4275-8217-dc6a541551b2 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: 5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:28.907090 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-812cf081-e739-4072-b681-1017bb113ca5 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:28.908002 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-812cf081-e739-4072-b681-1017bb113ca5 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] [instance: 0dc14c92-6364-4981-81c3-1c00e4f8187b] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:28.911880 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-45201128-e5f2-4275-8217-dc6a541551b2 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:28.912580 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-45201128-e5f2-4275-8217-dc6a541551b2 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:28.913166 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-45201128-e5f2-4275-8217-dc6a541551b2 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:28.914028 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-45201128-e5f2-4275-8217-dc6a541551b2 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:28.914734 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-45201128-e5f2-4275-8217-dc6a541551b2 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:28.915305 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-45201128-e5f2-4275-8217-dc6a541551b2 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:28.915862 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-812cf081-e739-4072-b681-1017bb113ca5 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:28.916444 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-812cf081-e739-4072-b681-1017bb113ca5 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:28.917041 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-812cf081-e739-4072-b681-1017bb113ca5 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:28.918328 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-812cf081-e739-4072-b681-1017bb113ca5 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:28.918885 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-812cf081-e739-4072-b681-1017bb113ca5 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:28.919488 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-812cf081-e739-4072-b681-1017bb113ca5 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:28.990648 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-812cf081-e739-4072-b681-1017bb113ca5 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc8294d49c654c8f9d8f7539b359df85', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27fc853acd7b4ef3803aec20a793bd06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:29.012065 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-812cf081-e739-4072-b681-1017bb113ca5 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] 10.222.0.16 "GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b" status: 200 len: 1273 microversion: 2.1 time: 0.110068 Oct 30 00:59:29.012650 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 190/368] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:28 2022] GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b => generated 1273 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:29.066105 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-84732bd0-ba18-4067-bb28-4ea154433a67 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:29.066939 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-84732bd0-ba18-4067-bb28-4ea154433a67 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:29.076971 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-84732bd0-ba18-4067-bb28-4ea154433a67 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:29.077771 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-84732bd0-ba18-4067-bb28-4ea154433a67 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:29.078500 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-84732bd0-ba18-4067-bb28-4ea154433a67 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:29.079301 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-84732bd0-ba18-4067-bb28-4ea154433a67 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:29.079993 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-84732bd0-ba18-4067-bb28-4ea154433a67 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:29.080664 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-84732bd0-ba18-4067-bb28-4ea154433a67 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:29.238694 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-84732bd0-ba18-4067-bb28-4ea154433a67 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:29.248483 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-84732bd0-ba18-4067-bb28-4ea154433a67 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1522 microversion: 2.1 time: 0.186143 Oct 30 00:59:29.249806 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 191/369] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:29 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1522 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:29.267977 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-45201128-e5f2-4275-8217-dc6a541551b2 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "POST /compute/v2.1/servers/5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.381345 Oct 30 00:59:29.268596 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 179/370] 10.222.0.16 () {60 vars in 1323 bytes} [Sun Oct 30 00:59:28 2022] POST /compute/v2.1/servers/5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a/os-volume_attachments => generated 194 bytes in 383 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:59:29.706888 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-b70fc3d4-153b-4ac9-bb23-da92de551426 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:29.707522 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-b70fc3d4-153b-4ac9-bb23-da92de551426 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 6eaaf253-16e7-4c62-a434-a4b8f5d7b536] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:29.713632 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b70fc3d4-153b-4ac9-bb23-da92de551426 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:29.714126 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b70fc3d4-153b-4ac9-bb23-da92de551426 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:29.714626 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b70fc3d4-153b-4ac9-bb23-da92de551426 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:29.715266 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b70fc3d4-153b-4ac9-bb23-da92de551426 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:29.715736 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b70fc3d4-153b-4ac9-bb23-da92de551426 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:29.716165 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b70fc3d4-153b-4ac9-bb23-da92de551426 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:29.731091 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.wsgi [None req-b70fc3d4-153b-4ac9-bb23-da92de551426 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] HTTP exception thrown: Instance 6eaaf253-16e7-4c62-a434-a4b8f5d7b536 could not be found. Oct 30 00:59:29.732973 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-b70fc3d4-153b-4ac9-bb23-da92de551426 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Request method failure captured: Oct 30 00:59:29.732973 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: request: GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536 HTTP/1.1 Oct 30 00:59:29.732973 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Accept: application/json Oct 30 00:59:29.732973 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Accept-Encoding: identity Oct 30 00:59:29.732973 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Connection: close Oct 30 00:59:29.732973 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Content-Length: 0 Oct 30 00:59:29.732973 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Content-Type: application/json Oct 30 00:59:29.732973 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Host: 10.222.0.16 Oct 30 00:59:29.732973 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Openstack-System-Scope: None Oct 30 00:59:29.732973 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: User-Agent: python-urllib3/1.26.12 Oct 30 00:59:29.732973 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Auth-Token: gAAAAABjXcxCc44LIEkKQ8a2UgGE4iirtf50DZU_H2glHE_HYBb_16OiP1wRwxnOeJW3GwZjrUbbKDl0gsmhkj628rcdOi7OiIsT6j3y617poGDGiePAXyZW0RN-KEA3m-VSK5uKK5FPKJqItmTZocqlRENpl9CFC31MgcWiu3nv_w45Yb558mg Oct 30 00:59:29.732973 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Domain-Id: None Oct 30 00:59:29.732973 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Domain-Name: None Oct 30 00:59:29.732973 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Identity-Status: Confirmed Oct 30 00:59:29.732973 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Is-Admin-Project: True Oct 30 00:59:29.732973 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Domain-Id: default Oct 30 00:59:29.732973 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Domain-Name: Default Oct 30 00:59:29.732973 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Id: dd16277cbc344c9aa8ffbc45ce1a16d2 Oct 30 00:59:29.732973 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Name: tempest-VolumesActionsTest-1212931503 Oct 30 00:59:29.732973 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Role: reader,member Oct 30 00:59:29.732973 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Roles: reader,member Oct 30 00:59:29.734472 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_dd16277cbc344c9aa8ffbc45ce1a16d2"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/dd16277cbc344c9aa8ffbc45ce1a16d2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/dd16277cbc344c9aa8ffbc45ce1a16d2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/dd16277cbc344c9aa8ffbc45ce1a16d2"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 00:59:29.734472 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant: tempest-VolumesActionsTest-1212931503 Oct 30 00:59:29.734472 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant-Id: dd16277cbc344c9aa8ffbc45ce1a16d2 Oct 30 00:59:29.734472 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant-Name: tempest-VolumesActionsTest-1212931503 Oct 30 00:59:29.734472 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User: tempest-VolumesActionsTest-1212931503-project Oct 30 00:59:29.734472 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Domain-Id: default Oct 30 00:59:29.734472 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Domain-Name: Default Oct 30 00:59:29.734472 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Id: 5f198d5e999b487bad75a6b423fe1adc Oct 30 00:59:29.734472 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Name: tempest-VolumesActionsTest-1212931503-project Oct 30 00:59:29.734472 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: method: > Oct 30 00:59:29.734472 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: exception: Instance 6eaaf253-16e7-4c62-a434-a4b8f5d7b536 could not be found. Oct 30 00:59:29.734472 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: action_args: {'id': '6eaaf253-16e7-4c62-a434-a4b8f5d7b536'} Oct 30 00:59:29.735366 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 00:59:29.735366 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 00:59:29.735366 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 30 00:59:29.735366 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 30 00:59:29.735366 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 30 00:59:29.735366 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 30 00:59:29.735366 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 30 00:59:29.735366 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 30 00:59:29.735366 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 30 00:59:29.735366 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 30 00:59:29.735366 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6eaaf253-16e7-4c62-a434-a4b8f5d7b536 could not be found. Oct 30 00:59:29.735366 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 00:59:29.735366 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 00:59:29.735366 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 00:59:29.735366 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 00:59:29.735366 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 00:59:29.735366 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 00:59:29.735366 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 00:59:29.735366 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 00:59:29.735366 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 00:59:29.736808 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 00:59:29.736808 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 30 00:59:29.736808 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 30 00:59:29.736808 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 30 00:59:29.736808 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 30 00:59:29.736808 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 30 00:59:29.736808 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 30 00:59:29.736808 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6eaaf253-16e7-4c62-a434-a4b8f5d7b536 could not be found. Oct 30 00:59:29.736808 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 00:59:29.736808 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 00:59:29.736808 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 00:59:29.736808 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 00:59:29.736808 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 00:59:29.736808 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 00:59:29.736808 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 00:59:29.736808 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 00:59:29.736808 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6eaaf253-16e7-4c62-a434-a4b8f5d7b536 could not be found. Oct 30 00:59:29.736808 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 00:59:29.738066 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-b70fc3d4-153b-4ac9-bb23-da92de551426 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Returning 404 to user: Instance 6eaaf253-16e7-4c62-a434-a4b8f5d7b536 could not be found. {{(pid=118961) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 00:59:29.738066 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-b70fc3d4-153b-4ac9-bb23-da92de551426 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536" status: 404 len: 111 microversion: 2.1 time: 0.031655 Oct 30 00:59:29.738066 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 192/371] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:29 2022] GET /compute/v2.1/servers/6eaaf253-16e7-4c62-a434-a4b8f5d7b536 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 30 00:59:29.845366 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-95f3188a-ffb3-4450-9159-88320ece0ec6 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-1920945144", "imageRef": "4c16f4f6-84cb-4463-b82c-b7c65f3826f0", "flavorRef": "452", "networks": [{"uuid": "e79f970e-5030-4a3e-9305-668e891be9fd"}]}} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:59:29.903243 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.network.neutron [None req-95f3188a-ffb3-4450-9159-88320ece0ec6 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] validate_networks() for [('e79f970e-5030-4a3e-9305-668e891be9fd', None, None, None, None, None)] {{(pid=118962) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 30 00:59:29.960059 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 30 00:59:29.960660 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Sun Oct 30 00:59:29 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.16) !!! Oct 30 00:59:30.039400 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-7fc94ad8-ab33-4a21-ab88-d2efb5600770 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:30.040055 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-7fc94ad8-ab33-4a21-ab88-d2efb5600770 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] [instance: 0dc14c92-6364-4981-81c3-1c00e4f8187b] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:30.043538 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 30 00:59:30.043741 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Sun Oct 30 00:59:30 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b (ip 10.222.0.16) !!! Oct 30 00:59:30.058165 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7fc94ad8-ab33-4a21-ab88-d2efb5600770 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:30.058671 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7fc94ad8-ab33-4a21-ab88-d2efb5600770 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:30.059086 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7fc94ad8-ab33-4a21-ab88-d2efb5600770 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:30.060047 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7fc94ad8-ab33-4a21-ab88-d2efb5600770 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:30.060287 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7fc94ad8-ab33-4a21-ab88-d2efb5600770 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:30.060690 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7fc94ad8-ab33-4a21-ab88-d2efb5600770 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:30.339682 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-7fc94ad8-ab33-4a21-ab88-d2efb5600770 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc8294d49c654c8f9d8f7539b359df85', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27fc853acd7b4ef3803aec20a793bd06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:30.355186 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-7fc94ad8-ab33-4a21-ab88-d2efb5600770 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] 10.222.0.16 "GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b" status: 200 len: 1315 microversion: 2.1 time: 0.321952 Oct 30 00:59:30.355780 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 193/372] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:30 2022] GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b => generated 1315 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:30.376644 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-875dbf27-1ab6-49a9-81ad-65647f2d17e4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:30.377268 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-875dbf27-1ab6-49a9-81ad-65647f2d17e4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:30.383411 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-875dbf27-1ab6-49a9-81ad-65647f2d17e4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:30.383979 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-875dbf27-1ab6-49a9-81ad-65647f2d17e4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:30.384575 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-875dbf27-1ab6-49a9-81ad-65647f2d17e4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:30.385465 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-875dbf27-1ab6-49a9-81ad-65647f2d17e4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:30.386057 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-875dbf27-1ab6-49a9-81ad-65647f2d17e4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:30.386788 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-875dbf27-1ab6-49a9-81ad-65647f2d17e4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:30.400426 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.virt.hardware [None req-95f3188a-ffb3-4450-9159-88320ece0ec6 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Flavor limits 0:0:0 {{(pid=118962) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 30 00:59:30.401123 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.virt.hardware [None req-95f3188a-ffb3-4450-9159-88320ece0ec6 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Image limits 0:0:0 {{(pid=118962) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 30 00:59:30.401751 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.virt.hardware [None req-95f3188a-ffb3-4450-9159-88320ece0ec6 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Flavor pref 0:0:0 {{(pid=118962) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 30 00:59:30.402301 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.virt.hardware [None req-95f3188a-ffb3-4450-9159-88320ece0ec6 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Image pref 0:0:0 {{(pid=118962) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 30 00:59:30.402837 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.virt.hardware [None req-95f3188a-ffb3-4450-9159-88320ece0ec6 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118962) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 30 00:59:30.413252 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-95f3188a-ffb3-4450-9159-88320ece0ec6 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:30.414451 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-95f3188a-ffb3-4450-9159-88320ece0ec6 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:30.415031 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-95f3188a-ffb3-4450-9159-88320ece0ec6 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:30.416517 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-95f3188a-ffb3-4450-9159-88320ece0ec6 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:30.417113 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-95f3188a-ffb3-4450-9159-88320ece0ec6 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:30.417693 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-95f3188a-ffb3-4450-9159-88320ece0ec6 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:30.439790 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.quota [None req-95f3188a-ffb3-4450-9159-88320ece0ec6 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Getting quotas for project dd16277cbc344c9aa8ffbc45ce1a16d2. Resources: {'ram', 'instances', 'cores'} {{(pid=118962) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 30 00:59:30.446013 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.quota [None req-95f3188a-ffb3-4450-9159-88320ece0ec6 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Getting quotas for user 5f198d5e999b487bad75a6b423fe1adc and project dd16277cbc344c9aa8ffbc45ce1a16d2. Resources: {'ram', 'instances', 'cores'} {{(pid=118962) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 30 00:59:30.458742 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-95f3188a-ffb3-4450-9159-88320ece0ec6 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Going to run 1 instances... {{(pid=118962) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 30 00:59:30.461627 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-95f3188a-ffb3-4450-9159-88320ece0ec6 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 3f2f7c97-5868-4aab-af09-dcdd578f5aed] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4c16f4f6-84cb-4463-b82c-b7c65f3826f0',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118962) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 30 00:59:30.509709 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-95f3188a-ffb3-4450-9159-88320ece0ec6 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.669496 Oct 30 00:59:30.510609 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 180/373] 10.222.0.16 () {60 vars in 1147 bytes} [Sun Oct 30 00:59:29 2022] POST /compute/v2.1/servers => generated 380 bytes in 671 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 30 00:59:30.540562 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-1a575f35-08a2-4548-92aa-737afec1c00a tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:30.541187 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-1a575f35-08a2-4548-92aa-737afec1c00a tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 3f2f7c97-5868-4aab-af09-dcdd578f5aed] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:30.542169 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-875dbf27-1ab6-49a9-81ad-65647f2d17e4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:30.551090 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-875dbf27-1ab6-49a9-81ad-65647f2d17e4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1522 microversion: 2.1 time: 0.181493 Oct 30 00:59:30.551766 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 194/374] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:30 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1522 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:30.597970 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-1a575f35-08a2-4548-92aa-737afec1c00a tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:30.605069 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-1a575f35-08a2-4548-92aa-737afec1c00a tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed" status: 200 len: 1214 microversion: 2.1 time: 0.076247 Oct 30 00:59:30.605641 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 181/375] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:30 2022] GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed => generated 1214 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:30.631198 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [req-8a486153-1cf9-4777-a08b-91a299572da1 req-cb40f934-4b52-4423-958f-316c7dd97d5c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0dc14c92-6364-4981-81c3-1c00e4f8187b", "tag": "5edec0bf-c7b4-4cb7-b191-28078b9dccca"}]} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:59:30.640647 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-8a486153-1cf9-4777-a08b-91a299572da1 req-cb40f934-4b52-4423-958f-316c7dd97d5c service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:30.641118 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-8a486153-1cf9-4777-a08b-91a299572da1 req-cb40f934-4b52-4423-958f-316c7dd97d5c service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:30.641538 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-8a486153-1cf9-4777-a08b-91a299572da1 req-cb40f934-4b52-4423-958f-316c7dd97d5c service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:30.654782 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.compute.server_external_events [req-8a486153-1cf9-4777-a08b-91a299572da1 req-cb40f934-4b52-4423-958f-316c7dd97d5c service nova] Creating event network-changed:5edec0bf-c7b4-4cb7-b191-28078b9dccca for instance 0dc14c92-6364-4981-81c3-1c00e4f8187b on hv-cinder-maste Oct 30 00:59:30.660318 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [req-8a486153-1cf9-4777-a08b-91a299572da1 req-cb40f934-4b52-4423-958f-316c7dd97d5c service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032656 Oct 30 00:59:30.660911 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 195/376] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 00:59:30 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:59:31.382879 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-c1a96782-1244-4c35-ad22-07e951ac4958 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:31.383491 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-c1a96782-1244-4c35-ad22-07e951ac4958 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] [instance: 0dc14c92-6364-4981-81c3-1c00e4f8187b] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:31.391208 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c1a96782-1244-4c35-ad22-07e951ac4958 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:31.391691 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c1a96782-1244-4c35-ad22-07e951ac4958 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:31.392152 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c1a96782-1244-4c35-ad22-07e951ac4958 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:31.392842 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c1a96782-1244-4c35-ad22-07e951ac4958 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:31.393293 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c1a96782-1244-4c35-ad22-07e951ac4958 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:31.393710 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c1a96782-1244-4c35-ad22-07e951ac4958 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:31.526704 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-c1a96782-1244-4c35-ad22-07e951ac4958 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc8294d49c654c8f9d8f7539b359df85', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27fc853acd7b4ef3803aec20a793bd06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:31.534836 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-c1a96782-1244-4c35-ad22-07e951ac4958 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] 10.222.0.16 "GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b" status: 200 len: 1315 microversion: 2.1 time: 0.155979 Oct 30 00:59:31.535412 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 182/377] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:31 2022] GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b => generated 1315 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:31.575393 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-30903fc8-d2f5-4496-87da-7b88954d2488 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:31.576127 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-30903fc8-d2f5-4496-87da-7b88954d2488 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:31.582824 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-30903fc8-d2f5-4496-87da-7b88954d2488 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:31.583302 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-30903fc8-d2f5-4496-87da-7b88954d2488 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:31.583747 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-30903fc8-d2f5-4496-87da-7b88954d2488 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:31.584447 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-30903fc8-d2f5-4496-87da-7b88954d2488 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:31.584891 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-30903fc8-d2f5-4496-87da-7b88954d2488 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:31.585308 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-30903fc8-d2f5-4496-87da-7b88954d2488 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:31.628345 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-ab70d510-2c26-4a19-8e1c-44110086b1e7 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:31.628860 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-ab70d510-2c26-4a19-8e1c-44110086b1e7 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 3f2f7c97-5868-4aab-af09-dcdd578f5aed] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:31.634807 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ab70d510-2c26-4a19-8e1c-44110086b1e7 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:31.635278 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ab70d510-2c26-4a19-8e1c-44110086b1e7 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:31.635700 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ab70d510-2c26-4a19-8e1c-44110086b1e7 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:31.636403 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ab70d510-2c26-4a19-8e1c-44110086b1e7 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:31.636851 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ab70d510-2c26-4a19-8e1c-44110086b1e7 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:31.637253 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ab70d510-2c26-4a19-8e1c-44110086b1e7 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:31.707245 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-ab70d510-2c26-4a19-8e1c-44110086b1e7 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:31.715169 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-ab70d510-2c26-4a19-8e1c-44110086b1e7 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed" status: 200 len: 1218 microversion: 2.1 time: 0.090742 Oct 30 00:59:31.715746 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 183/378] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:31 2022] GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed => generated 1218 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 00:59:31.737500 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-30903fc8-d2f5-4496-87da-7b88954d2488 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:31.745438 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-30903fc8-d2f5-4496-87da-7b88954d2488 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1522 microversion: 2.1 time: 0.174661 Oct 30 00:59:31.745996 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 196/379] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:31 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1522 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:32.558300 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-04f09dee-c026-493e-a1be-8a9f4b182eae tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:32.558919 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-04f09dee-c026-493e-a1be-8a9f4b182eae tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] [instance: 0dc14c92-6364-4981-81c3-1c00e4f8187b] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:32.566792 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-04f09dee-c026-493e-a1be-8a9f4b182eae tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:32.567295 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-04f09dee-c026-493e-a1be-8a9f4b182eae tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:32.567770 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-04f09dee-c026-493e-a1be-8a9f4b182eae tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:32.568446 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-04f09dee-c026-493e-a1be-8a9f4b182eae tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:32.568896 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-04f09dee-c026-493e-a1be-8a9f4b182eae tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:32.569318 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-04f09dee-c026-493e-a1be-8a9f4b182eae tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:32.707738 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-04f09dee-c026-493e-a1be-8a9f4b182eae tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc8294d49c654c8f9d8f7539b359df85', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27fc853acd7b4ef3803aec20a793bd06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:32.715744 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-04f09dee-c026-493e-a1be-8a9f4b182eae tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] 10.222.0.16 "GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b" status: 200 len: 1315 microversion: 2.1 time: 0.161285 Oct 30 00:59:32.716327 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 184/380] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:32 2022] GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b => generated 1315 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:32.738566 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-9abf5618-c5dc-4be0-83f1-41d6e9551bc4 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:32.739175 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-9abf5618-c5dc-4be0-83f1-41d6e9551bc4 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 3f2f7c97-5868-4aab-af09-dcdd578f5aed] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:32.745612 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9abf5618-c5dc-4be0-83f1-41d6e9551bc4 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:32.746097 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9abf5618-c5dc-4be0-83f1-41d6e9551bc4 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:32.746535 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9abf5618-c5dc-4be0-83f1-41d6e9551bc4 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:32.747254 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9abf5618-c5dc-4be0-83f1-41d6e9551bc4 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:32.747883 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9abf5618-c5dc-4be0-83f1-41d6e9551bc4 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:32.748313 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9abf5618-c5dc-4be0-83f1-41d6e9551bc4 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:32.767167 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:59:32.769002 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-816b7e11-6e8e-4f3f-901f-836a543ed0b3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:32.769689 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-816b7e11-6e8e-4f3f-901f-836a543ed0b3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:32.776253 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-816b7e11-6e8e-4f3f-901f-836a543ed0b3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:32.776738 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-816b7e11-6e8e-4f3f-901f-836a543ed0b3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:32.777165 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-816b7e11-6e8e-4f3f-901f-836a543ed0b3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:32.777868 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-816b7e11-6e8e-4f3f-901f-836a543ed0b3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:32.778319 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-816b7e11-6e8e-4f3f-901f-836a543ed0b3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:32.778737 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-816b7e11-6e8e-4f3f-901f-836a543ed0b3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:32.812356 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-9abf5618-c5dc-4be0-83f1-41d6e9551bc4 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:32.820310 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-9abf5618-c5dc-4be0-83f1-41d6e9551bc4 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed" status: 200 len: 1218 microversion: 2.1 time: 0.085626 Oct 30 00:59:32.820989 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 197/381] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:32 2022] GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed => generated 1218 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:32.904893 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-816b7e11-6e8e-4f3f-901f-836a543ed0b3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:32.913164 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-816b7e11-6e8e-4f3f-901f-836a543ed0b3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1522 microversion: 2.1 time: 0.148060 Oct 30 00:59:32.913728 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 185/382] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:32 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1522 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:33.740597 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-b455a81f-3117-4345-be84-3449df4d7ab1 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:33.741442 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-b455a81f-3117-4345-be84-3449df4d7ab1 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] [instance: 0dc14c92-6364-4981-81c3-1c00e4f8187b] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:33.750956 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b455a81f-3117-4345-be84-3449df4d7ab1 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:33.751647 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b455a81f-3117-4345-be84-3449df4d7ab1 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:33.752934 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b455a81f-3117-4345-be84-3449df4d7ab1 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:33.753312 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b455a81f-3117-4345-be84-3449df4d7ab1 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:33.753947 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b455a81f-3117-4345-be84-3449df4d7ab1 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:33.754549 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b455a81f-3117-4345-be84-3449df4d7ab1 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:33.846017 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-ef85694b-db9e-479b-bdb0-6bfa13b17198 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:33.847390 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-ef85694b-db9e-479b-bdb0-6bfa13b17198 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 3f2f7c97-5868-4aab-af09-dcdd578f5aed] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:33.855917 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ef85694b-db9e-479b-bdb0-6bfa13b17198 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:33.856723 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ef85694b-db9e-479b-bdb0-6bfa13b17198 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:33.857498 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ef85694b-db9e-479b-bdb0-6bfa13b17198 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:33.858524 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ef85694b-db9e-479b-bdb0-6bfa13b17198 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:33.859197 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ef85694b-db9e-479b-bdb0-6bfa13b17198 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:33.859951 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ef85694b-db9e-479b-bdb0-6bfa13b17198 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:33.915878 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-b455a81f-3117-4345-be84-3449df4d7ab1 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc8294d49c654c8f9d8f7539b359df85', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27fc853acd7b4ef3803aec20a793bd06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:33.925115 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-b455a81f-3117-4345-be84-3449df4d7ab1 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] 10.222.0.16 "GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b" status: 200 len: 1315 microversion: 2.1 time: 0.190025 Oct 30 00:59:33.926355 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 198/383] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:33 2022] GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b => generated 1315 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:33.935855 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-7a56a145-acb9-4d8d-ab25-048f24d67122 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:33.936709 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-7a56a145-acb9-4d8d-ab25-048f24d67122 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:33.943527 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7a56a145-acb9-4d8d-ab25-048f24d67122 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:33.944041 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7a56a145-acb9-4d8d-ab25-048f24d67122 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:33.944516 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7a56a145-acb9-4d8d-ab25-048f24d67122 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:33.945398 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7a56a145-acb9-4d8d-ab25-048f24d67122 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:33.945528 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-ef85694b-db9e-479b-bdb0-6bfa13b17198 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:33.945839 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7a56a145-acb9-4d8d-ab25-048f24d67122 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:33.946407 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7a56a145-acb9-4d8d-ab25-048f24d67122 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:33.953821 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-ef85694b-db9e-479b-bdb0-6bfa13b17198 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed" status: 200 len: 1218 microversion: 2.1 time: 0.112851 Oct 30 00:59:33.954632 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 186/384] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:33 2022] GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed => generated 1218 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:34.115682 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-7a56a145-acb9-4d8d-ab25-048f24d67122 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:34.123965 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-7a56a145-acb9-4d8d-ab25-048f24d67122 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1522 microversion: 2.1 time: 0.191901 Oct 30 00:59:34.127659 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 199/385] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:33 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1522 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 00:59:34.953307 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-7c8db248-b05a-46f1-adbd-0e6caa7a5553 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:34.954363 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-7c8db248-b05a-46f1-adbd-0e6caa7a5553 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] [instance: 0dc14c92-6364-4981-81c3-1c00e4f8187b] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:34.965207 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7c8db248-b05a-46f1-adbd-0e6caa7a5553 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:34.966001 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7c8db248-b05a-46f1-adbd-0e6caa7a5553 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:34.966706 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7c8db248-b05a-46f1-adbd-0e6caa7a5553 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:34.968198 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7c8db248-b05a-46f1-adbd-0e6caa7a5553 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:34.968755 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7c8db248-b05a-46f1-adbd-0e6caa7a5553 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:34.969198 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7c8db248-b05a-46f1-adbd-0e6caa7a5553 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:34.976886 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-fdc3d8e7-bfe7-45c7-acf4-3bc2dba87c21 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:34.977568 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-fdc3d8e7-bfe7-45c7-acf4-3bc2dba87c21 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 3f2f7c97-5868-4aab-af09-dcdd578f5aed] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:34.984022 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-fdc3d8e7-bfe7-45c7-acf4-3bc2dba87c21 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:34.984606 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-fdc3d8e7-bfe7-45c7-acf4-3bc2dba87c21 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:34.985040 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-fdc3d8e7-bfe7-45c7-acf4-3bc2dba87c21 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:34.985728 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-fdc3d8e7-bfe7-45c7-acf4-3bc2dba87c21 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:34.986174 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-fdc3d8e7-bfe7-45c7-acf4-3bc2dba87c21 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:34.986588 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-fdc3d8e7-bfe7-45c7-acf4-3bc2dba87c21 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:35.078896 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-fdc3d8e7-bfe7-45c7-acf4-3bc2dba87c21 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:35.087263 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-fdc3d8e7-bfe7-45c7-acf4-3bc2dba87c21 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed" status: 200 len: 1266 microversion: 2.1 time: 0.114184 Oct 30 00:59:35.088879 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 200/386] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:34 2022] GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed => generated 1266 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:35.127468 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-7c8db248-b05a-46f1-adbd-0e6caa7a5553 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc8294d49c654c8f9d8f7539b359df85', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27fc853acd7b4ef3803aec20a793bd06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:35.130895 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell1:SELECT=96,nova_cell1:INSERT=2,nova_cell1:UPDATE=2 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:59:35.136023 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-7c8db248-b05a-46f1-adbd-0e6caa7a5553 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] 10.222.0.16 "GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b" status: 200 len: 1315 microversion: 2.1 time: 0.189768 Oct 30 00:59:35.136586 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 187/387] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:34 2022] GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b => generated 1315 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:35.149011 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-7ed2b194-fae4-4d60-99aa-fb785ed50180 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:35.149760 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-7ed2b194-fae4-4d60-99aa-fb785ed50180 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:35.163269 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7ed2b194-fae4-4d60-99aa-fb785ed50180 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:35.163785 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7ed2b194-fae4-4d60-99aa-fb785ed50180 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:35.164285 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7ed2b194-fae4-4d60-99aa-fb785ed50180 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:35.165147 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7ed2b194-fae4-4d60-99aa-fb785ed50180 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:35.165755 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7ed2b194-fae4-4d60-99aa-fb785ed50180 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:35.166178 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7ed2b194-fae4-4d60-99aa-fb785ed50180 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:35.306448 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-7ed2b194-fae4-4d60-99aa-fb785ed50180 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:35.314632 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-7ed2b194-fae4-4d60-99aa-fb785ed50180 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1522 microversion: 2.1 time: 0.169743 Oct 30 00:59:35.315187 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 201/388] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:35 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1522 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:35.771892 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [req-9879b3ed-5b07-4676-adbc-a15c0778c6a6 req-c62d2f87-6b98-4ec7-ac3d-e7c029f10b46 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0dc14c92-6364-4981-81c3-1c00e4f8187b", "name": "network-vif-plugged", "status": "completed", "tag": "5edec0bf-c7b4-4cb7-b191-28078b9dccca"}]} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:59:35.786611 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-9879b3ed-5b07-4676-adbc-a15c0778c6a6 req-c62d2f87-6b98-4ec7-ac3d-e7c029f10b46 service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:35.787225 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-9879b3ed-5b07-4676-adbc-a15c0778c6a6 req-c62d2f87-6b98-4ec7-ac3d-e7c029f10b46 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:35.787764 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-9879b3ed-5b07-4676-adbc-a15c0778c6a6 req-c62d2f87-6b98-4ec7-ac3d-e7c029f10b46 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:35.800152 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.compute.server_external_events [req-9879b3ed-5b07-4676-adbc-a15c0778c6a6 req-c62d2f87-6b98-4ec7-ac3d-e7c029f10b46 service nova] Creating event network-vif-plugged:5edec0bf-c7b4-4cb7-b191-28078b9dccca for instance 0dc14c92-6364-4981-81c3-1c00e4f8187b on hv-cinder-maste Oct 30 00:59:35.807241 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [req-9879b3ed-5b07-4676-adbc-a15c0778c6a6 req-c62d2f87-6b98-4ec7-ac3d-e7c029f10b46 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040459 Oct 30 00:59:35.808181 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 188/389] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 00:59:35 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:59:36.112333 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-2533e1da-fe06-4b51-b394-e8b47fd922b6 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:36.112953 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-2533e1da-fe06-4b51-b394-e8b47fd922b6 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 3f2f7c97-5868-4aab-af09-dcdd578f5aed] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:36.119410 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2533e1da-fe06-4b51-b394-e8b47fd922b6 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:36.119947 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2533e1da-fe06-4b51-b394-e8b47fd922b6 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:36.120402 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2533e1da-fe06-4b51-b394-e8b47fd922b6 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:36.121107 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2533e1da-fe06-4b51-b394-e8b47fd922b6 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:36.121592 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2533e1da-fe06-4b51-b394-e8b47fd922b6 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:36.122019 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2533e1da-fe06-4b51-b394-e8b47fd922b6 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:36.165435 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-ec958dba-4290-4e85-b157-88c31338bc98 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:36.166421 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-ec958dba-4290-4e85-b157-88c31338bc98 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] [instance: 0dc14c92-6364-4981-81c3-1c00e4f8187b] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:36.176184 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ec958dba-4290-4e85-b157-88c31338bc98 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:36.176915 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ec958dba-4290-4e85-b157-88c31338bc98 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:36.177467 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ec958dba-4290-4e85-b157-88c31338bc98 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:36.178274 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ec958dba-4290-4e85-b157-88c31338bc98 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:36.178817 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ec958dba-4290-4e85-b157-88c31338bc98 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:36.179328 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ec958dba-4290-4e85-b157-88c31338bc98 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:36.329830 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-2533e1da-fe06-4b51-b394-e8b47fd922b6 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:36.340205 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-2533e1da-fe06-4b51-b394-e8b47fd922b6 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed" status: 200 len: 1314 microversion: 2.1 time: 0.231872 Oct 30 00:59:36.341158 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 202/390] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:36 2022] GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed => generated 1314 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:36.353341 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-9e038bd0-11eb-47f9-b60a-aaaddfe8345b tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:36.354324 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-9e038bd0-11eb-47f9-b60a-aaaddfe8345b tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:36.362907 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9e038bd0-11eb-47f9-b60a-aaaddfe8345b tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:36.364352 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9e038bd0-11eb-47f9-b60a-aaaddfe8345b tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:36.365554 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9e038bd0-11eb-47f9-b60a-aaaddfe8345b tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:36.366621 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9e038bd0-11eb-47f9-b60a-aaaddfe8345b tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:36.367239 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9e038bd0-11eb-47f9-b60a-aaaddfe8345b tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:36.368513 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9e038bd0-11eb-47f9-b60a-aaaddfe8345b tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:36.373243 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-ec958dba-4290-4e85-b157-88c31338bc98 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc8294d49c654c8f9d8f7539b359df85', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27fc853acd7b4ef3803aec20a793bd06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:36.381842 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-ec958dba-4290-4e85-b157-88c31338bc98 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] 10.222.0.16 "GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b" status: 200 len: 1315 microversion: 2.1 time: 0.222646 Oct 30 00:59:36.382511 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 189/391] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:36 2022] GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b => generated 1315 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:36.504623 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-9e038bd0-11eb-47f9-b60a-aaaddfe8345b tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:36.512953 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-9e038bd0-11eb-47f9-b60a-aaaddfe8345b tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1522 microversion: 2.1 time: 0.164457 Oct 30 00:59:36.513688 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 203/392] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:36 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1522 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:36.807785 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [req-02bb0ab7-c550-4c15-a5ed-ddcf48dc07ff req-3f798b10-a5c8-481d-b478-29fa6afd0ee7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3f2f7c97-5868-4aab-af09-dcdd578f5aed", "tag": "642e374e-315b-4436-8c38-4da776b5b51d"}]} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:59:36.810445 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-02bb0ab7-c550-4c15-a5ed-ddcf48dc07ff req-3f798b10-a5c8-481d-b478-29fa6afd0ee7 service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:36.810939 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-02bb0ab7-c550-4c15-a5ed-ddcf48dc07ff req-3f798b10-a5c8-481d-b478-29fa6afd0ee7 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:36.811366 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-02bb0ab7-c550-4c15-a5ed-ddcf48dc07ff req-3f798b10-a5c8-481d-b478-29fa6afd0ee7 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:36.823169 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.compute.server_external_events [req-02bb0ab7-c550-4c15-a5ed-ddcf48dc07ff req-3f798b10-a5c8-481d-b478-29fa6afd0ee7 service nova] Creating event network-changed:642e374e-315b-4436-8c38-4da776b5b51d for instance 3f2f7c97-5868-4aab-af09-dcdd578f5aed on hv-cinder-maste Oct 30 00:59:36.828703 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [req-02bb0ab7-c550-4c15-a5ed-ddcf48dc07ff req-3f798b10-a5c8-481d-b478-29fa6afd0ee7 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032947 Oct 30 00:59:36.829663 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 190/393] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 00:59:36 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:59:37.368609 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-e92c1efa-b358-4e13-ba0d-172d354c178c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:37.369626 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-e92c1efa-b358-4e13-ba0d-172d354c178c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 3f2f7c97-5868-4aab-af09-dcdd578f5aed] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:37.378546 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e92c1efa-b358-4e13-ba0d-172d354c178c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:37.379035 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e92c1efa-b358-4e13-ba0d-172d354c178c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:37.379476 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e92c1efa-b358-4e13-ba0d-172d354c178c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:37.380217 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e92c1efa-b358-4e13-ba0d-172d354c178c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:37.380665 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e92c1efa-b358-4e13-ba0d-172d354c178c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:37.381090 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e92c1efa-b358-4e13-ba0d-172d354c178c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:37.406870 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-175fabbe-434a-4dff-9ac8-863fdb62641f tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:37.407510 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-175fabbe-434a-4dff-9ac8-863fdb62641f tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] [instance: 0dc14c92-6364-4981-81c3-1c00e4f8187b] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:37.413814 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-175fabbe-434a-4dff-9ac8-863fdb62641f tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:37.414382 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-175fabbe-434a-4dff-9ac8-863fdb62641f tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:37.414858 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-175fabbe-434a-4dff-9ac8-863fdb62641f tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:37.415552 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-175fabbe-434a-4dff-9ac8-863fdb62641f tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:37.416093 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-175fabbe-434a-4dff-9ac8-863fdb62641f tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:37.416512 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-175fabbe-434a-4dff-9ac8-863fdb62641f tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:37.588712 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-e92c1efa-b358-4e13-ba0d-172d354c178c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:37.591237 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-175fabbe-434a-4dff-9ac8-863fdb62641f tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc8294d49c654c8f9d8f7539b359df85', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27fc853acd7b4ef3803aec20a793bd06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:37.596589 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-e92c1efa-b358-4e13-ba0d-172d354c178c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed" status: 200 len: 1314 microversion: 2.1 time: 0.234425 Oct 30 00:59:37.597140 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 204/394] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:37 2022] GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed => generated 1314 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:37.599136 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-175fabbe-434a-4dff-9ac8-863fdb62641f tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] 10.222.0.16 "GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b" status: 200 len: 1315 microversion: 2.1 time: 0.197626 Oct 30 00:59:37.599683 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 191/395] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:37 2022] GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b => generated 1315 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 00:59:37.601782 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-f2cf7a40-5c76-45b0-a331-1c55056fc5c2 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:37.602379 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-f2cf7a40-5c76-45b0-a331-1c55056fc5c2 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:37.608192 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f2cf7a40-5c76-45b0-a331-1c55056fc5c2 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:37.608601 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f2cf7a40-5c76-45b0-a331-1c55056fc5c2 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:37.609030 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f2cf7a40-5c76-45b0-a331-1c55056fc5c2 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:37.609714 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f2cf7a40-5c76-45b0-a331-1c55056fc5c2 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:37.610210 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f2cf7a40-5c76-45b0-a331-1c55056fc5c2 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:37.610621 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f2cf7a40-5c76-45b0-a331-1c55056fc5c2 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:37.750711 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-f2cf7a40-5c76-45b0-a331-1c55056fc5c2 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:37.758857 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-f2cf7a40-5c76-45b0-a331-1c55056fc5c2 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1522 microversion: 2.1 time: 0.160526 Oct 30 00:59:37.759454 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 205/396] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:37 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1522 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:38.620183 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-9e78e3d0-e36d-42ef-8f0f-e73f3b02980a tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:38.620819 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-9e78e3d0-e36d-42ef-8f0f-e73f3b02980a tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 3f2f7c97-5868-4aab-af09-dcdd578f5aed] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:38.622492 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-14f9dd6f-1653-4fea-b8b2-86274507e40e tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:38.623084 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-14f9dd6f-1653-4fea-b8b2-86274507e40e tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] [instance: 0dc14c92-6364-4981-81c3-1c00e4f8187b] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:38.627819 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9e78e3d0-e36d-42ef-8f0f-e73f3b02980a tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:38.628295 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9e78e3d0-e36d-42ef-8f0f-e73f3b02980a tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:38.628726 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9e78e3d0-e36d-42ef-8f0f-e73f3b02980a tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:38.629431 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-14f9dd6f-1653-4fea-b8b2-86274507e40e tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:38.629715 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9e78e3d0-e36d-42ef-8f0f-e73f3b02980a tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:38.629919 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-14f9dd6f-1653-4fea-b8b2-86274507e40e tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:38.630170 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9e78e3d0-e36d-42ef-8f0f-e73f3b02980a tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:38.630343 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-14f9dd6f-1653-4fea-b8b2-86274507e40e tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:38.630590 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9e78e3d0-e36d-42ef-8f0f-e73f3b02980a tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:38.631029 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-14f9dd6f-1653-4fea-b8b2-86274507e40e tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:38.631473 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-14f9dd6f-1653-4fea-b8b2-86274507e40e tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:38.631926 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-14f9dd6f-1653-4fea-b8b2-86274507e40e tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:38.834213 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-9e78e3d0-e36d-42ef-8f0f-e73f3b02980a tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:38.837322 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-14f9dd6f-1653-4fea-b8b2-86274507e40e tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc8294d49c654c8f9d8f7539b359df85', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27fc853acd7b4ef3803aec20a793bd06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:38.842938 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-9e78e3d0-e36d-42ef-8f0f-e73f3b02980a tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed" status: 200 len: 1314 microversion: 2.1 time: 0.226720 Oct 30 00:59:38.843501 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 192/397] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:38 2022] GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed => generated 1314 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:38.845388 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-14f9dd6f-1653-4fea-b8b2-86274507e40e tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] 10.222.0.16 "GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b" status: 200 len: 1315 microversion: 2.1 time: 0.226679 Oct 30 00:59:38.846010 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 206/398] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:38 2022] GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b => generated 1315 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:38.856717 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-24d9bbfa-69c9-4760-8695-dd7220614724 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:38.856717 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-24d9bbfa-69c9-4760-8695-dd7220614724 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:38.856717 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-24d9bbfa-69c9-4760-8695-dd7220614724 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:38.856717 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-24d9bbfa-69c9-4760-8695-dd7220614724 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:38.857056 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-24d9bbfa-69c9-4760-8695-dd7220614724 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:38.857056 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-24d9bbfa-69c9-4760-8695-dd7220614724 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:38.857056 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-24d9bbfa-69c9-4760-8695-dd7220614724 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:38.857056 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-24d9bbfa-69c9-4760-8695-dd7220614724 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:39.000463 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-24d9bbfa-69c9-4760-8695-dd7220614724 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:39.008928 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-24d9bbfa-69c9-4760-8695-dd7220614724 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1522 microversion: 2.1 time: 0.164286 Oct 30 00:59:39.009472 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 193/399] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:38 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1522 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:39.867938 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-0d0eabd0-8a5c-455c-8dcd-4a340f7867ef tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:39.868504 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-78430c3f-4f73-4642-a1e6-842cfc5ddb5e tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:39.869103 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-0d0eabd0-8a5c-455c-8dcd-4a340f7867ef tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 3f2f7c97-5868-4aab-af09-dcdd578f5aed] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:39.869840 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-78430c3f-4f73-4642-a1e6-842cfc5ddb5e tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] [instance: 0dc14c92-6364-4981-81c3-1c00e4f8187b] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:39.876333 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0d0eabd0-8a5c-455c-8dcd-4a340f7867ef tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:39.876945 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0d0eabd0-8a5c-455c-8dcd-4a340f7867ef tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:39.877244 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-78430c3f-4f73-4642-a1e6-842cfc5ddb5e tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:39.877687 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0d0eabd0-8a5c-455c-8dcd-4a340f7867ef tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:39.877896 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-78430c3f-4f73-4642-a1e6-842cfc5ddb5e tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:39.878398 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0d0eabd0-8a5c-455c-8dcd-4a340f7867ef tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:39.878560 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-78430c3f-4f73-4642-a1e6-842cfc5ddb5e tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:39.879197 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0d0eabd0-8a5c-455c-8dcd-4a340f7867ef tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:39.879457 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-78430c3f-4f73-4642-a1e6-842cfc5ddb5e tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:39.879618 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0d0eabd0-8a5c-455c-8dcd-4a340f7867ef tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:39.880279 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-78430c3f-4f73-4642-a1e6-842cfc5ddb5e tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:39.880972 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-78430c3f-4f73-4642-a1e6-842cfc5ddb5e tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:40.082771 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-0d0eabd0-8a5c-455c-8dcd-4a340f7867ef tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:40.085827 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-78430c3f-4f73-4642-a1e6-842cfc5ddb5e tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc8294d49c654c8f9d8f7539b359df85', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27fc853acd7b4ef3803aec20a793bd06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:40.090828 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-0d0eabd0-8a5c-455c-8dcd-4a340f7867ef tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed" status: 200 len: 1314 microversion: 2.1 time: 0.227731 Oct 30 00:59:40.091450 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 207/400] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:39 2022] GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed => generated 1314 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:40.094364 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-78430c3f-4f73-4642-a1e6-842cfc5ddb5e tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] 10.222.0.16 "GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b" status: 200 len: 1464 microversion: 2.1 time: 0.229877 Oct 30 00:59:40.095006 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 194/401] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:39 2022] GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b => generated 1464 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:40.103634 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-d57dc84c-cb97-4367-b8fe-e6a283d82276 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:40.103634 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-d57dc84c-cb97-4367-b8fe-e6a283d82276 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:40.103634 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d57dc84c-cb97-4367-b8fe-e6a283d82276 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:40.103634 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d57dc84c-cb97-4367-b8fe-e6a283d82276 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:40.104171 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d57dc84c-cb97-4367-b8fe-e6a283d82276 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:40.104593 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d57dc84c-cb97-4367-b8fe-e6a283d82276 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:40.105008 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d57dc84c-cb97-4367-b8fe-e6a283d82276 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:40.105839 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d57dc84c-cb97-4367-b8fe-e6a283d82276 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:40.149337 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-697f9f54-f616-45f0-a6fa-fb785066f307 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:40.149940 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-697f9f54-f616-45f0-a6fa-fb785066f307 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] [instance: 0dc14c92-6364-4981-81c3-1c00e4f8187b] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:40.157325 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-697f9f54-f616-45f0-a6fa-fb785066f307 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:40.157914 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-697f9f54-f616-45f0-a6fa-fb785066f307 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:40.158407 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-697f9f54-f616-45f0-a6fa-fb785066f307 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:40.159156 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-697f9f54-f616-45f0-a6fa-fb785066f307 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:40.159743 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-697f9f54-f616-45f0-a6fa-fb785066f307 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:40.160329 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-697f9f54-f616-45f0-a6fa-fb785066f307 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:40.189498 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-697f9f54-f616-45f0-a6fa-fb785066f307 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] [instance: 0dc14c92-6364-4981-81c3-1c00e4f8187b] Going to try to terminate instance {{(pid=118962) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 30 00:59:40.253063 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-d57dc84c-cb97-4367-b8fe-e6a283d82276 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:40.262224 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-d57dc84c-cb97-4367-b8fe-e6a283d82276 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1522 microversion: 2.1 time: 0.169747 Oct 30 00:59:40.262821 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 208/402] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:40 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1522 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:40.282600 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-697f9f54-f616-45f0-a6fa-fb785066f307 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] 10.222.0.16 "DELETE /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b" status: 204 len: 0 microversion: 2.1 time: 0.137225 Oct 30 00:59:40.283249 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 195/403] 10.222.0.16 () {58 vars in 1239 bytes} [Sun Oct 30 00:59:40 2022] DELETE /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b => generated 0 bytes in 139 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Oct 30 00:59:40.292017 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-5add5431-508c-464c-bf8c-503b15fc1086 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:40.292643 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-5add5431-508c-464c-bf8c-503b15fc1086 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] [instance: 0dc14c92-6364-4981-81c3-1c00e4f8187b] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:40.301425 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5add5431-508c-464c-bf8c-503b15fc1086 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:40.302116 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5add5431-508c-464c-bf8c-503b15fc1086 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:40.302835 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5add5431-508c-464c-bf8c-503b15fc1086 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:40.304063 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5add5431-508c-464c-bf8c-503b15fc1086 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:40.304783 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5add5431-508c-464c-bf8c-503b15fc1086 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:40.305604 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5add5431-508c-464c-bf8c-503b15fc1086 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:40.472752 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-5add5431-508c-464c-bf8c-503b15fc1086 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc8294d49c654c8f9d8f7539b359df85', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27fc853acd7b4ef3803aec20a793bd06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:40.483763 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-5add5431-508c-464c-bf8c-503b15fc1086 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] 10.222.0.16 "GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b" status: 200 len: 1470 microversion: 2.1 time: 0.196062 Oct 30 00:59:40.484347 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 209/404] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:40 2022] GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b => generated 1470 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:41.115528 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-206c7aea-0088-4a22-962e-48577c453cc7 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:41.116258 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-206c7aea-0088-4a22-962e-48577c453cc7 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 3f2f7c97-5868-4aab-af09-dcdd578f5aed] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:41.118193 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell0:SELECT=4 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:59:41.118862 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell0:SELECT=16 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:59:41.124981 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-206c7aea-0088-4a22-962e-48577c453cc7 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:41.125472 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-206c7aea-0088-4a22-962e-48577c453cc7 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:41.125885 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-206c7aea-0088-4a22-962e-48577c453cc7 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:41.126808 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-206c7aea-0088-4a22-962e-48577c453cc7 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:41.127253 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-206c7aea-0088-4a22-962e-48577c453cc7 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:41.127669 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-206c7aea-0088-4a22-962e-48577c453cc7 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:41.273449 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-206c7aea-0088-4a22-962e-48577c453cc7 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:41.284097 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-206c7aea-0088-4a22-962e-48577c453cc7 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed" status: 200 len: 1314 microversion: 2.1 time: 0.172685 Oct 30 00:59:41.285225 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 196/405] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:41 2022] GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed => generated 1314 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:41.289043 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-37ca8fce-98cc-4fd6-ac70-15ff9b5eff18 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:41.289657 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-37ca8fce-98cc-4fd6-ac70-15ff9b5eff18 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:41.296008 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-37ca8fce-98cc-4fd6-ac70-15ff9b5eff18 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:41.296496 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-37ca8fce-98cc-4fd6-ac70-15ff9b5eff18 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:41.296985 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-37ca8fce-98cc-4fd6-ac70-15ff9b5eff18 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:41.297688 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-37ca8fce-98cc-4fd6-ac70-15ff9b5eff18 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:41.298139 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-37ca8fce-98cc-4fd6-ac70-15ff9b5eff18 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:41.298557 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-37ca8fce-98cc-4fd6-ac70-15ff9b5eff18 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:41.442020 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-37ca8fce-98cc-4fd6-ac70-15ff9b5eff18 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:41.450361 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-37ca8fce-98cc-4fd6-ac70-15ff9b5eff18 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1522 microversion: 2.1 time: 0.165145 Oct 30 00:59:41.450931 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 210/406] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:41 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1522 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:41.516171 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-8c37db00-a9f7-41a7-999e-6441da00ae06 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:41.517166 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-8c37db00-a9f7-41a7-999e-6441da00ae06 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] [instance: 0dc14c92-6364-4981-81c3-1c00e4f8187b] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:41.526696 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8c37db00-a9f7-41a7-999e-6441da00ae06 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:41.527478 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8c37db00-a9f7-41a7-999e-6441da00ae06 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:41.528217 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8c37db00-a9f7-41a7-999e-6441da00ae06 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:41.529374 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8c37db00-a9f7-41a7-999e-6441da00ae06 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:41.530147 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8c37db00-a9f7-41a7-999e-6441da00ae06 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:41.530819 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8c37db00-a9f7-41a7-999e-6441da00ae06 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:41.669987 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-8c37db00-a9f7-41a7-999e-6441da00ae06 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc8294d49c654c8f9d8f7539b359df85', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27fc853acd7b4ef3803aec20a793bd06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:41.681998 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-8c37db00-a9f7-41a7-999e-6441da00ae06 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] 10.222.0.16 "GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b" status: 200 len: 1470 microversion: 2.1 time: 0.171757 Oct 30 00:59:41.682584 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 197/407] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:41 2022] GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b => generated 1470 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:42.311630 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-b2ffb3a7-322a-4eec-bb21-ead32c55d075 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:42.312695 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-b2ffb3a7-322a-4eec-bb21-ead32c55d075 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 3f2f7c97-5868-4aab-af09-dcdd578f5aed] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:42.319799 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b2ffb3a7-322a-4eec-bb21-ead32c55d075 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:42.320292 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b2ffb3a7-322a-4eec-bb21-ead32c55d075 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:42.320697 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b2ffb3a7-322a-4eec-bb21-ead32c55d075 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:42.321418 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b2ffb3a7-322a-4eec-bb21-ead32c55d075 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:42.321860 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b2ffb3a7-322a-4eec-bb21-ead32c55d075 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:42.322282 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b2ffb3a7-322a-4eec-bb21-ead32c55d075 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:42.453367 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-b2ffb3a7-322a-4eec-bb21-ead32c55d075 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:42.461062 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-b2ffb3a7-322a-4eec-bb21-ead32c55d075 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed" status: 200 len: 1314 microversion: 2.1 time: 0.156044 Oct 30 00:59:42.461631 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 211/408] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:42 2022] GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed => generated 1314 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:42.474572 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-47729263-4ee5-4da0-a2ca-992555f4511b tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:42.475110 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-47729263-4ee5-4da0-a2ca-992555f4511b tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:42.482102 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-47729263-4ee5-4da0-a2ca-992555f4511b tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:42.482577 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-47729263-4ee5-4da0-a2ca-992555f4511b tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:42.483006 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-47729263-4ee5-4da0-a2ca-992555f4511b tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:42.483685 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-47729263-4ee5-4da0-a2ca-992555f4511b tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:42.484156 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-47729263-4ee5-4da0-a2ca-992555f4511b tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:42.484572 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-47729263-4ee5-4da0-a2ca-992555f4511b tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:42.614625 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-47729263-4ee5-4da0-a2ca-992555f4511b tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:42.622498 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-47729263-4ee5-4da0-a2ca-992555f4511b tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1522 microversion: 2.1 time: 0.152596 Oct 30 00:59:42.623077 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 198/409] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:42 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1522 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:42.711546 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-91329695-b537-4d24-99bd-8b21ad930514 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:42.712393 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-91329695-b537-4d24-99bd-8b21ad930514 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] [instance: 0dc14c92-6364-4981-81c3-1c00e4f8187b] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:42.720781 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-91329695-b537-4d24-99bd-8b21ad930514 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:42.721419 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-91329695-b537-4d24-99bd-8b21ad930514 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:42.722180 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-91329695-b537-4d24-99bd-8b21ad930514 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:42.723089 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-91329695-b537-4d24-99bd-8b21ad930514 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:42.723681 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-91329695-b537-4d24-99bd-8b21ad930514 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:42.724290 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-91329695-b537-4d24-99bd-8b21ad930514 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:42.862146 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-91329695-b537-4d24-99bd-8b21ad930514 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc8294d49c654c8f9d8f7539b359df85', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27fc853acd7b4ef3803aec20a793bd06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:42.870768 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-91329695-b537-4d24-99bd-8b21ad930514 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] 10.222.0.16 "GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b" status: 200 len: 1470 microversion: 2.1 time: 0.164493 Oct 30 00:59:42.871416 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 212/410] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:42 2022] GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b => generated 1470 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:43.486657 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-711081be-e364-4505-8ed9-70180482b74c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:43.487356 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-711081be-e364-4505-8ed9-70180482b74c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 3f2f7c97-5868-4aab-af09-dcdd578f5aed] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:43.493794 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-711081be-e364-4505-8ed9-70180482b74c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:43.494295 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-711081be-e364-4505-8ed9-70180482b74c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:43.494714 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-711081be-e364-4505-8ed9-70180482b74c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:43.495409 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-711081be-e364-4505-8ed9-70180482b74c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:43.495886 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-711081be-e364-4505-8ed9-70180482b74c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:43.496284 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-711081be-e364-4505-8ed9-70180482b74c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:43.630415 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-711081be-e364-4505-8ed9-70180482b74c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:43.638729 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-711081be-e364-4505-8ed9-70180482b74c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed" status: 200 len: 1314 microversion: 2.1 time: 0.156082 Oct 30 00:59:43.639285 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 199/411] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:43 2022] GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed => generated 1314 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:43.646248 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-dab613a3-db82-4734-bdc3-a0f13607afdb tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:43.646962 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-dab613a3-db82-4734-bdc3-a0f13607afdb tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:43.653121 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-dab613a3-db82-4734-bdc3-a0f13607afdb tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:43.653606 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-dab613a3-db82-4734-bdc3-a0f13607afdb tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:43.654034 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-dab613a3-db82-4734-bdc3-a0f13607afdb tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:43.654721 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-dab613a3-db82-4734-bdc3-a0f13607afdb tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:43.655166 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-dab613a3-db82-4734-bdc3-a0f13607afdb tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:43.655579 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-dab613a3-db82-4734-bdc3-a0f13607afdb tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:43.795023 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-dab613a3-db82-4734-bdc3-a0f13607afdb tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:43.802964 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-dab613a3-db82-4734-bdc3-a0f13607afdb tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1522 microversion: 2.1 time: 0.160919 Oct 30 00:59:43.803548 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 213/412] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:43 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1522 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:43.897147 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-c2239552-11af-44c8-abee-fc053dbf228f tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:43.897964 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-c2239552-11af-44c8-abee-fc053dbf228f tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] [instance: 0dc14c92-6364-4981-81c3-1c00e4f8187b] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:43.900654 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_api:SELECT=93,nova_api:INSERT=6,nova_api:UPDATE=1 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:59:43.908852 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c2239552-11af-44c8-abee-fc053dbf228f tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:43.909688 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c2239552-11af-44c8-abee-fc053dbf228f tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:43.910103 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c2239552-11af-44c8-abee-fc053dbf228f tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:43.910810 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c2239552-11af-44c8-abee-fc053dbf228f tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:43.911252 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c2239552-11af-44c8-abee-fc053dbf228f tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:43.911670 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c2239552-11af-44c8-abee-fc053dbf228f tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:44.042803 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-c2239552-11af-44c8-abee-fc053dbf228f tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc8294d49c654c8f9d8f7539b359df85', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27fc853acd7b4ef3803aec20a793bd06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:44.054607 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-c2239552-11af-44c8-abee-fc053dbf228f tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] 10.222.0.16 "GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b" status: 200 len: 1470 microversion: 2.1 time: 0.163252 Oct 30 00:59:44.055315 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 200/413] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:43 2022] GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b => generated 1470 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:44.666605 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-278526b0-ce5c-4aac-b703-947bf85d86ab tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:44.667437 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-278526b0-ce5c-4aac-b703-947bf85d86ab tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 3f2f7c97-5868-4aab-af09-dcdd578f5aed] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:44.677859 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-278526b0-ce5c-4aac-b703-947bf85d86ab tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:44.678699 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-278526b0-ce5c-4aac-b703-947bf85d86ab tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:44.679423 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-278526b0-ce5c-4aac-b703-947bf85d86ab tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:44.680745 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-278526b0-ce5c-4aac-b703-947bf85d86ab tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:44.681528 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-278526b0-ce5c-4aac-b703-947bf85d86ab tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:44.682242 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-278526b0-ce5c-4aac-b703-947bf85d86ab tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:44.688740 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell1:SELECT=100 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:59:44.817149 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-278526b0-ce5c-4aac-b703-947bf85d86ab tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:44.829307 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-25695a1e-3a0d-46ca-a60d-0a713550b341 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:44.829824 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-278526b0-ce5c-4aac-b703-947bf85d86ab tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed" status: 200 len: 1314 microversion: 2.1 time: 0.169477 Oct 30 00:59:44.829918 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-25695a1e-3a0d-46ca-a60d-0a713550b341 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:44.830835 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 214/414] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:44 2022] GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed => generated 1314 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:44.836823 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-25695a1e-3a0d-46ca-a60d-0a713550b341 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:44.837297 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-25695a1e-3a0d-46ca-a60d-0a713550b341 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:44.837722 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-25695a1e-3a0d-46ca-a60d-0a713550b341 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:44.838402 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-25695a1e-3a0d-46ca-a60d-0a713550b341 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:44.838847 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-25695a1e-3a0d-46ca-a60d-0a713550b341 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:44.839239 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-25695a1e-3a0d-46ca-a60d-0a713550b341 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:44.967888 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-25695a1e-3a0d-46ca-a60d-0a713550b341 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:44.976213 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-25695a1e-3a0d-46ca-a60d-0a713550b341 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1522 microversion: 2.1 time: 0.153252 Oct 30 00:59:44.976777 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 201/415] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:44 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1522 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:45.085012 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-1fb56935-743b-46ac-9ee3-fdd73436373e tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:45.085685 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-1fb56935-743b-46ac-9ee3-fdd73436373e tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] [instance: 0dc14c92-6364-4981-81c3-1c00e4f8187b] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:45.092632 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1fb56935-743b-46ac-9ee3-fdd73436373e tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:45.093194 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1fb56935-743b-46ac-9ee3-fdd73436373e tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:45.093676 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1fb56935-743b-46ac-9ee3-fdd73436373e tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:45.094442 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1fb56935-743b-46ac-9ee3-fdd73436373e tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:45.094931 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1fb56935-743b-46ac-9ee3-fdd73436373e tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:45.095393 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1fb56935-743b-46ac-9ee3-fdd73436373e tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:45.227768 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-1fb56935-743b-46ac-9ee3-fdd73436373e tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc8294d49c654c8f9d8f7539b359df85', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27fc853acd7b4ef3803aec20a793bd06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:45.237186 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-1fb56935-743b-46ac-9ee3-fdd73436373e tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] 10.222.0.16 "GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b" status: 200 len: 1470 microversion: 2.1 time: 0.157584 Oct 30 00:59:45.237801 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 215/416] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:45 2022] GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b => generated 1470 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:45.856272 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-20b2551b-76f1-4143-b1f6-99197f812a35 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:45.857264 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-20b2551b-76f1-4143-b1f6-99197f812a35 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 3f2f7c97-5868-4aab-af09-dcdd578f5aed] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:45.866777 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-20b2551b-76f1-4143-b1f6-99197f812a35 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:45.867256 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-20b2551b-76f1-4143-b1f6-99197f812a35 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:45.867679 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-20b2551b-76f1-4143-b1f6-99197f812a35 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:45.868396 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-20b2551b-76f1-4143-b1f6-99197f812a35 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:45.868841 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-20b2551b-76f1-4143-b1f6-99197f812a35 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:45.869222 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-20b2551b-76f1-4143-b1f6-99197f812a35 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:45.999811 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-67c58ff3-06e3-42f1-855b-c92a8035d4b4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:46.000391 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-67c58ff3-06e3-42f1-855b-c92a8035d4b4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:46.006955 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-67c58ff3-06e3-42f1-855b-c92a8035d4b4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:46.007434 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-67c58ff3-06e3-42f1-855b-c92a8035d4b4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:46.007898 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-67c58ff3-06e3-42f1-855b-c92a8035d4b4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:46.008584 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-67c58ff3-06e3-42f1-855b-c92a8035d4b4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:46.009057 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-67c58ff3-06e3-42f1-855b-c92a8035d4b4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:46.009482 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-67c58ff3-06e3-42f1-855b-c92a8035d4b4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:46.012789 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-20b2551b-76f1-4143-b1f6-99197f812a35 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:46.020709 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-20b2551b-76f1-4143-b1f6-99197f812a35 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed" status: 200 len: 1314 microversion: 2.1 time: 0.170882 Oct 30 00:59:46.021282 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 202/417] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:45 2022] GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed => generated 1314 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:46.139787 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-67c58ff3-06e3-42f1-855b-c92a8035d4b4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:46.151436 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-67c58ff3-06e3-42f1-855b-c92a8035d4b4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1522 microversion: 2.1 time: 0.155595 Oct 30 00:59:46.152290 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 216/418] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:45 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1522 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:46.264818 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-eba10fcb-13ee-4da9-b133-2a5909c6a39d tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:46.266006 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-eba10fcb-13ee-4da9-b133-2a5909c6a39d tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] [instance: 0dc14c92-6364-4981-81c3-1c00e4f8187b] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:46.276156 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-eba10fcb-13ee-4da9-b133-2a5909c6a39d tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:46.277036 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-eba10fcb-13ee-4da9-b133-2a5909c6a39d tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:46.277788 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-eba10fcb-13ee-4da9-b133-2a5909c6a39d tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:46.278937 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-eba10fcb-13ee-4da9-b133-2a5909c6a39d tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:46.279757 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-eba10fcb-13ee-4da9-b133-2a5909c6a39d tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:46.280525 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-eba10fcb-13ee-4da9-b133-2a5909c6a39d tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:46.431436 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-eba10fcb-13ee-4da9-b133-2a5909c6a39d tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc8294d49c654c8f9d8f7539b359df85', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27fc853acd7b4ef3803aec20a793bd06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:46.439687 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-eba10fcb-13ee-4da9-b133-2a5909c6a39d tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] 10.222.0.16 "GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b" status: 200 len: 1470 microversion: 2.1 time: 0.181385 Oct 30 00:59:46.440294 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 203/419] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:46 2022] GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b => generated 1470 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:47.048198 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-20af2aa0-b85d-4598-ac17-1e8cb8a71d52 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:47.049209 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-20af2aa0-b85d-4598-ac17-1e8cb8a71d52 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 3f2f7c97-5868-4aab-af09-dcdd578f5aed] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:47.056286 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-20af2aa0-b85d-4598-ac17-1e8cb8a71d52 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:47.056772 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-20af2aa0-b85d-4598-ac17-1e8cb8a71d52 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:47.057209 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-20af2aa0-b85d-4598-ac17-1e8cb8a71d52 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:47.057901 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-20af2aa0-b85d-4598-ac17-1e8cb8a71d52 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:47.058351 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-20af2aa0-b85d-4598-ac17-1e8cb8a71d52 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:47.058770 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-20af2aa0-b85d-4598-ac17-1e8cb8a71d52 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:47.178240 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-24c763a9-de30-4efa-8847-bdcfb4fe8d21 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:47.178846 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-24c763a9-de30-4efa-8847-bdcfb4fe8d21 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:47.185199 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-24c763a9-de30-4efa-8847-bdcfb4fe8d21 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:47.185672 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-24c763a9-de30-4efa-8847-bdcfb4fe8d21 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:47.186094 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-24c763a9-de30-4efa-8847-bdcfb4fe8d21 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:47.186775 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-24c763a9-de30-4efa-8847-bdcfb4fe8d21 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:47.187235 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-24c763a9-de30-4efa-8847-bdcfb4fe8d21 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:47.187642 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-24c763a9-de30-4efa-8847-bdcfb4fe8d21 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:47.188893 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-20af2aa0-b85d-4598-ac17-1e8cb8a71d52 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:47.191755 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:59:47.197092 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-20af2aa0-b85d-4598-ac17-1e8cb8a71d52 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed" status: 200 len: 1314 microversion: 2.1 time: 0.155880 Oct 30 00:59:47.197666 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 217/420] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:47 2022] GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed => generated 1314 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:47.321814 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-24c763a9-de30-4efa-8847-bdcfb4fe8d21 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:47.331435 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-24c763a9-de30-4efa-8847-bdcfb4fe8d21 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1522 microversion: 2.1 time: 0.157484 Oct 30 00:59:47.332120 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 204/421] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:47 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1522 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:47.467638 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-b5cda641-71e9-44c5-9b65-cbecac85a464 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:47.468776 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-b5cda641-71e9-44c5-9b65-cbecac85a464 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] [instance: 0dc14c92-6364-4981-81c3-1c00e4f8187b] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:47.476932 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b5cda641-71e9-44c5-9b65-cbecac85a464 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:47.477425 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b5cda641-71e9-44c5-9b65-cbecac85a464 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:47.477859 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b5cda641-71e9-44c5-9b65-cbecac85a464 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:47.478552 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b5cda641-71e9-44c5-9b65-cbecac85a464 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:47.478997 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b5cda641-71e9-44c5-9b65-cbecac85a464 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:47.479419 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b5cda641-71e9-44c5-9b65-cbecac85a464 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:47.613119 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-b5cda641-71e9-44c5-9b65-cbecac85a464 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc8294d49c654c8f9d8f7539b359df85', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27fc853acd7b4ef3803aec20a793bd06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:47.621210 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-b5cda641-71e9-44c5-9b65-cbecac85a464 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] 10.222.0.16 "GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b" status: 200 len: 1470 microversion: 2.1 time: 0.160063 Oct 30 00:59:47.621753 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 218/422] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:47 2022] GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b => generated 1470 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:48.224462 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-092b6858-ec34-4656-bcd5-28e2793890df tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:48.225050 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-092b6858-ec34-4656-bcd5-28e2793890df tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 3f2f7c97-5868-4aab-af09-dcdd578f5aed] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:48.231770 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-092b6858-ec34-4656-bcd5-28e2793890df tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:48.232274 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-092b6858-ec34-4656-bcd5-28e2793890df tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:48.232703 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-092b6858-ec34-4656-bcd5-28e2793890df tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:48.233402 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-092b6858-ec34-4656-bcd5-28e2793890df tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:48.233840 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-092b6858-ec34-4656-bcd5-28e2793890df tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:48.234250 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-092b6858-ec34-4656-bcd5-28e2793890df tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:48.353571 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-6c35f291-fc4c-4b70-bd5b-8c4b7c325338 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:48.354213 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-6c35f291-fc4c-4b70-bd5b-8c4b7c325338 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:48.360166 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6c35f291-fc4c-4b70-bd5b-8c4b7c325338 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:48.360743 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6c35f291-fc4c-4b70-bd5b-8c4b7c325338 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:48.361169 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6c35f291-fc4c-4b70-bd5b-8c4b7c325338 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:48.361852 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6c35f291-fc4c-4b70-bd5b-8c4b7c325338 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:48.365429 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-092b6858-ec34-4656-bcd5-28e2793890df tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:48.366236 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6c35f291-fc4c-4b70-bd5b-8c4b7c325338 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:48.366897 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6c35f291-fc4c-4b70-bd5b-8c4b7c325338 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:48.373391 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-092b6858-ec34-4656-bcd5-28e2793890df tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed" status: 200 len: 1314 microversion: 2.1 time: 0.154936 Oct 30 00:59:48.374141 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 205/423] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:48 2022] GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed => generated 1314 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:48.503402 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-6c35f291-fc4c-4b70-bd5b-8c4b7c325338 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:48.511543 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-6c35f291-fc4c-4b70-bd5b-8c4b7c325338 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1522 microversion: 2.1 time: 0.161866 Oct 30 00:59:48.512140 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 219/424] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:48 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1522 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 00:59:48.646481 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-a2553458-5d02-4487-9514-4479aac0c5e3 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:48.647099 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-a2553458-5d02-4487-9514-4479aac0c5e3 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] [instance: 0dc14c92-6364-4981-81c3-1c00e4f8187b] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:48.656291 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a2553458-5d02-4487-9514-4479aac0c5e3 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:48.656815 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a2553458-5d02-4487-9514-4479aac0c5e3 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:48.657340 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a2553458-5d02-4487-9514-4479aac0c5e3 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:48.658039 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a2553458-5d02-4487-9514-4479aac0c5e3 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:48.658616 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a2553458-5d02-4487-9514-4479aac0c5e3 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:48.659364 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a2553458-5d02-4487-9514-4479aac0c5e3 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:48.796822 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-a2553458-5d02-4487-9514-4479aac0c5e3 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc8294d49c654c8f9d8f7539b359df85', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27fc853acd7b4ef3803aec20a793bd06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:48.805071 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-a2553458-5d02-4487-9514-4479aac0c5e3 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] 10.222.0.16 "GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b" status: 200 len: 1470 microversion: 2.1 time: 0.162701 Oct 30 00:59:48.805651 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 206/425] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:48 2022] GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b => generated 1470 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:49.397522 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-26810141-ed68-4edc-988b-87d01950053c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:49.398144 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-26810141-ed68-4edc-988b-87d01950053c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 3f2f7c97-5868-4aab-af09-dcdd578f5aed] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:49.406015 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-26810141-ed68-4edc-988b-87d01950053c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:49.406504 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-26810141-ed68-4edc-988b-87d01950053c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:49.406938 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-26810141-ed68-4edc-988b-87d01950053c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:49.407640 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-26810141-ed68-4edc-988b-87d01950053c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:49.408123 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-26810141-ed68-4edc-988b-87d01950053c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:49.408546 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-26810141-ed68-4edc-988b-87d01950053c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:49.534400 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-0acf656e-ed86-4a78-b5f3-a489b575b85e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:49.535033 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-0acf656e-ed86-4a78-b5f3-a489b575b85e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:49.541868 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0acf656e-ed86-4a78-b5f3-a489b575b85e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:49.542349 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0acf656e-ed86-4a78-b5f3-a489b575b85e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:49.542506 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-26810141-ed68-4edc-988b-87d01950053c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:49.542773 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0acf656e-ed86-4a78-b5f3-a489b575b85e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:49.543465 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0acf656e-ed86-4a78-b5f3-a489b575b85e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:49.543965 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0acf656e-ed86-4a78-b5f3-a489b575b85e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:49.544357 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0acf656e-ed86-4a78-b5f3-a489b575b85e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:49.550866 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-26810141-ed68-4edc-988b-87d01950053c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed" status: 200 len: 1314 microversion: 2.1 time: 0.157385 Oct 30 00:59:49.551451 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 220/426] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:49 2022] GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed => generated 1314 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:49.674528 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-0acf656e-ed86-4a78-b5f3-a489b575b85e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:49.682706 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-0acf656e-ed86-4a78-b5f3-a489b575b85e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1522 microversion: 2.1 time: 0.152237 Oct 30 00:59:49.684594 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 207/427] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:49 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1522 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:49.830124 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-fc23c545-8cf7-40b2-8699-c5d8b31d498e tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:49.830750 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-fc23c545-8cf7-40b2-8699-c5d8b31d498e tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] [instance: 0dc14c92-6364-4981-81c3-1c00e4f8187b] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:49.839236 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-fc23c545-8cf7-40b2-8699-c5d8b31d498e tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:49.839742 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-fc23c545-8cf7-40b2-8699-c5d8b31d498e tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:49.840270 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-fc23c545-8cf7-40b2-8699-c5d8b31d498e tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:49.840956 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-fc23c545-8cf7-40b2-8699-c5d8b31d498e tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:49.841402 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-fc23c545-8cf7-40b2-8699-c5d8b31d498e tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:49.841826 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-fc23c545-8cf7-40b2-8699-c5d8b31d498e tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:49.977916 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-fc23c545-8cf7-40b2-8699-c5d8b31d498e tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc8294d49c654c8f9d8f7539b359df85', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27fc853acd7b4ef3803aec20a793bd06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:49.985690 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-fc23c545-8cf7-40b2-8699-c5d8b31d498e tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] 10.222.0.16 "GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b" status: 200 len: 1470 microversion: 2.1 time: 0.161177 Oct 30 00:59:49.986324 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 221/428] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:49 2022] GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b => generated 1470 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:50.574460 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-6b642cf9-81c5-4649-b5a8-ebeb4031b568 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:50.575390 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-6b642cf9-81c5-4649-b5a8-ebeb4031b568 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 3f2f7c97-5868-4aab-af09-dcdd578f5aed] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:50.582278 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6b642cf9-81c5-4649-b5a8-ebeb4031b568 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:50.582758 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6b642cf9-81c5-4649-b5a8-ebeb4031b568 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:50.583210 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6b642cf9-81c5-4649-b5a8-ebeb4031b568 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:50.583982 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6b642cf9-81c5-4649-b5a8-ebeb4031b568 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:50.584436 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6b642cf9-81c5-4649-b5a8-ebeb4031b568 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:50.584855 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6b642cf9-81c5-4649-b5a8-ebeb4031b568 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:50.707473 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-4aee576c-4331-450f-877b-ec3d5f2e58d3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:50.708205 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-4aee576c-4331-450f-877b-ec3d5f2e58d3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:50.714896 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4aee576c-4331-450f-877b-ec3d5f2e58d3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:50.715848 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4aee576c-4331-450f-877b-ec3d5f2e58d3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:50.716800 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4aee576c-4331-450f-877b-ec3d5f2e58d3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:50.717681 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4aee576c-4331-450f-877b-ec3d5f2e58d3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:50.718128 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4aee576c-4331-450f-877b-ec3d5f2e58d3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:50.718543 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4aee576c-4331-450f-877b-ec3d5f2e58d3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:50.723120 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-6b642cf9-81c5-4649-b5a8-ebeb4031b568 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:50.732348 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-6b642cf9-81c5-4649-b5a8-ebeb4031b568 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed" status: 200 len: 1314 microversion: 2.1 time: 0.161730 Oct 30 00:59:50.732988 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 208/429] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:50 2022] GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed => generated 1314 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:50.852964 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-4aee576c-4331-450f-877b-ec3d5f2e58d3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:50.860891 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-4aee576c-4331-450f-877b-ec3d5f2e58d3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1522 microversion: 2.1 time: 0.157928 Oct 30 00:59:50.861449 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 222/430] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:50 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1522 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 00:59:51.014252 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-2bc838c0-0a59-4058-928a-4dbafedd7755 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:51.015292 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-2bc838c0-0a59-4058-928a-4dbafedd7755 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] [instance: 0dc14c92-6364-4981-81c3-1c00e4f8187b] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:51.024084 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2bc838c0-0a59-4058-928a-4dbafedd7755 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:51.024568 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2bc838c0-0a59-4058-928a-4dbafedd7755 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:51.024995 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2bc838c0-0a59-4058-928a-4dbafedd7755 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:51.025684 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2bc838c0-0a59-4058-928a-4dbafedd7755 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:51.026127 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2bc838c0-0a59-4058-928a-4dbafedd7755 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:51.026544 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2bc838c0-0a59-4058-928a-4dbafedd7755 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:51.379633 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-2bc838c0-0a59-4058-928a-4dbafedd7755 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc8294d49c654c8f9d8f7539b359df85', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27fc853acd7b4ef3803aec20a793bd06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:51.387616 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-2bc838c0-0a59-4058-928a-4dbafedd7755 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] 10.222.0.16 "GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b" status: 200 len: 1470 microversion: 2.1 time: 0.380521 Oct 30 00:59:51.388242 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 209/431] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:51 2022] GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b => generated 1470 bytes in 382 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:51.759903 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-ed84225f-28d0-4e20-ac09-1b9fc663be9d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:51.760913 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-ed84225f-28d0-4e20-ac09-1b9fc663be9d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 3f2f7c97-5868-4aab-af09-dcdd578f5aed] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:51.770905 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ed84225f-28d0-4e20-ac09-1b9fc663be9d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:51.771770 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ed84225f-28d0-4e20-ac09-1b9fc663be9d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:51.772427 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ed84225f-28d0-4e20-ac09-1b9fc663be9d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:51.773326 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ed84225f-28d0-4e20-ac09-1b9fc663be9d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:51.773781 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ed84225f-28d0-4e20-ac09-1b9fc663be9d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:51.774201 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ed84225f-28d0-4e20-ac09-1b9fc663be9d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:51.885062 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-8158db8b-0c26-419d-9278-8ee3bb52e8b3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:51.885702 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-8158db8b-0c26-419d-9278-8ee3bb52e8b3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:51.892590 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8158db8b-0c26-419d-9278-8ee3bb52e8b3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:51.893047 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8158db8b-0c26-419d-9278-8ee3bb52e8b3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:51.893836 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8158db8b-0c26-419d-9278-8ee3bb52e8b3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:51.894694 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8158db8b-0c26-419d-9278-8ee3bb52e8b3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:51.895143 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8158db8b-0c26-419d-9278-8ee3bb52e8b3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:51.895555 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8158db8b-0c26-419d-9278-8ee3bb52e8b3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:51.908498 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-ed84225f-28d0-4e20-ac09-1b9fc663be9d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:51.916386 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-ed84225f-28d0-4e20-ac09-1b9fc663be9d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed" status: 200 len: 1314 microversion: 2.1 time: 0.162958 Oct 30 00:59:51.916949 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 223/432] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:51 2022] GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed => generated 1314 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:52.024755 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-8158db8b-0c26-419d-9278-8ee3bb52e8b3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:52.032635 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-8158db8b-0c26-419d-9278-8ee3bb52e8b3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1522 microversion: 2.1 time: 0.151726 Oct 30 00:59:52.033210 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 210/433] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:51 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1522 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:52.414654 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-dcb9844d-7c12-4322-81a3-f6ff64c86d69 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:52.415575 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-dcb9844d-7c12-4322-81a3-f6ff64c86d69 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] [instance: 0dc14c92-6364-4981-81c3-1c00e4f8187b] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:52.418460 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_api:SELECT=93,nova_api:UPDATE=1 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:59:52.426578 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-dcb9844d-7c12-4322-81a3-f6ff64c86d69 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:52.427360 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-dcb9844d-7c12-4322-81a3-f6ff64c86d69 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:52.428115 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-dcb9844d-7c12-4322-81a3-f6ff64c86d69 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:52.429196 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-dcb9844d-7c12-4322-81a3-f6ff64c86d69 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:52.429894 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-dcb9844d-7c12-4322-81a3-f6ff64c86d69 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:52.430401 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-dcb9844d-7c12-4322-81a3-f6ff64c86d69 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:52.569196 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-dcb9844d-7c12-4322-81a3-f6ff64c86d69 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc8294d49c654c8f9d8f7539b359df85', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27fc853acd7b4ef3803aec20a793bd06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:52.577489 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-dcb9844d-7c12-4322-81a3-f6ff64c86d69 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] 10.222.0.16 "GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b" status: 200 len: 1470 microversion: 2.1 time: 0.169017 Oct 30 00:59:52.578082 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 224/434] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:52 2022] GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b => generated 1470 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:52.939520 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-519a50a7-fd21-401d-8849-cbd7c939dc13 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:52.940128 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-519a50a7-fd21-401d-8849-cbd7c939dc13 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 3f2f7c97-5868-4aab-af09-dcdd578f5aed] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:52.947068 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-519a50a7-fd21-401d-8849-cbd7c939dc13 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:52.947545 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-519a50a7-fd21-401d-8849-cbd7c939dc13 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:52.948000 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-519a50a7-fd21-401d-8849-cbd7c939dc13 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:52.948680 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-519a50a7-fd21-401d-8849-cbd7c939dc13 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:52.949120 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-519a50a7-fd21-401d-8849-cbd7c939dc13 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:52.949530 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-519a50a7-fd21-401d-8849-cbd7c939dc13 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:53.056383 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-217f1cbf-c08a-4819-a36c-03b53cd034a3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:53.056973 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-217f1cbf-c08a-4819-a36c-03b53cd034a3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:53.063548 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-217f1cbf-c08a-4819-a36c-03b53cd034a3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:53.064068 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-217f1cbf-c08a-4819-a36c-03b53cd034a3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:53.064500 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-217f1cbf-c08a-4819-a36c-03b53cd034a3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:53.065201 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-217f1cbf-c08a-4819-a36c-03b53cd034a3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:53.065651 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-217f1cbf-c08a-4819-a36c-03b53cd034a3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:53.066070 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-217f1cbf-c08a-4819-a36c-03b53cd034a3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:53.087944 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-519a50a7-fd21-401d-8849-cbd7c939dc13 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:53.096194 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-519a50a7-fd21-401d-8849-cbd7c939dc13 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed" status: 200 len: 1314 microversion: 2.1 time: 0.160486 Oct 30 00:59:53.096973 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 211/435] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:52 2022] GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed => generated 1314 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:53.201428 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-217f1cbf-c08a-4819-a36c-03b53cd034a3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:53.213310 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-217f1cbf-c08a-4819-a36c-03b53cd034a3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1522 microversion: 2.1 time: 0.161290 Oct 30 00:59:53.213862 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 225/436] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:53 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1522 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:53.606868 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-64f5505e-51e8-4789-97d4-8082743962da tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:53.607638 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-64f5505e-51e8-4789-97d4-8082743962da tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] [instance: 0dc14c92-6364-4981-81c3-1c00e4f8187b] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:53.615858 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-64f5505e-51e8-4789-97d4-8082743962da tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:53.616488 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-64f5505e-51e8-4789-97d4-8082743962da tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:53.617381 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-64f5505e-51e8-4789-97d4-8082743962da tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:53.618463 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-64f5505e-51e8-4789-97d4-8082743962da tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:53.619060 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-64f5505e-51e8-4789-97d4-8082743962da tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:53.619613 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-64f5505e-51e8-4789-97d4-8082743962da tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:53.760283 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-64f5505e-51e8-4789-97d4-8082743962da tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc8294d49c654c8f9d8f7539b359df85', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27fc853acd7b4ef3803aec20a793bd06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:53.771879 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-64f5505e-51e8-4789-97d4-8082743962da tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] 10.222.0.16 "GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b" status: 200 len: 1470 microversion: 2.1 time: 0.170070 Oct 30 00:59:53.772597 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 212/437] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:53 2022] GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b => generated 1470 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:54.123539 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-87cba3d0-3467-4a6b-bd25-cccfeef23eb4 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:54.124587 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-87cba3d0-3467-4a6b-bd25-cccfeef23eb4 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 3f2f7c97-5868-4aab-af09-dcdd578f5aed] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:54.134092 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-87cba3d0-3467-4a6b-bd25-cccfeef23eb4 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:54.134568 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-87cba3d0-3467-4a6b-bd25-cccfeef23eb4 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:54.135066 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-87cba3d0-3467-4a6b-bd25-cccfeef23eb4 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:54.135751 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-87cba3d0-3467-4a6b-bd25-cccfeef23eb4 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:54.136219 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-87cba3d0-3467-4a6b-bd25-cccfeef23eb4 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:54.136631 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-87cba3d0-3467-4a6b-bd25-cccfeef23eb4 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:54.236770 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-74978ab2-820c-406e-a620-a568d10a1b99 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:54.237362 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-74978ab2-820c-406e-a620-a568d10a1b99 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:54.244331 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-74978ab2-820c-406e-a620-a568d10a1b99 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:54.244812 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-74978ab2-820c-406e-a620-a568d10a1b99 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:54.245335 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-74978ab2-820c-406e-a620-a568d10a1b99 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:54.246028 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-74978ab2-820c-406e-a620-a568d10a1b99 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:54.246475 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-74978ab2-820c-406e-a620-a568d10a1b99 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:54.246895 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-74978ab2-820c-406e-a620-a568d10a1b99 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:54.276069 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-87cba3d0-3467-4a6b-bd25-cccfeef23eb4 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:54.284204 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-87cba3d0-3467-4a6b-bd25-cccfeef23eb4 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed" status: 200 len: 1314 microversion: 2.1 time: 0.167149 Oct 30 00:59:54.284749 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 226/438] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:54 2022] GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed => generated 1314 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:54.374824 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-74978ab2-820c-406e-a620-a568d10a1b99 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:54.382674 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-74978ab2-820c-406e-a620-a568d10a1b99 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1522 microversion: 2.1 time: 0.149755 Oct 30 00:59:54.383327 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 213/439] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:54 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1522 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:54.799208 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-6f798ee5-ace7-4e74-877e-ec6b54f2c0e6 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:54.800255 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-6f798ee5-ace7-4e74-877e-ec6b54f2c0e6 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] [instance: 0dc14c92-6364-4981-81c3-1c00e4f8187b] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:54.808878 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6f798ee5-ace7-4e74-877e-ec6b54f2c0e6 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:54.809362 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6f798ee5-ace7-4e74-877e-ec6b54f2c0e6 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:54.809785 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6f798ee5-ace7-4e74-877e-ec6b54f2c0e6 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:54.810479 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6f798ee5-ace7-4e74-877e-ec6b54f2c0e6 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:54.810930 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6f798ee5-ace7-4e74-877e-ec6b54f2c0e6 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:54.811352 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6f798ee5-ace7-4e74-877e-ec6b54f2c0e6 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:54.944497 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-6f798ee5-ace7-4e74-877e-ec6b54f2c0e6 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc8294d49c654c8f9d8f7539b359df85', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27fc853acd7b4ef3803aec20a793bd06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:54.952748 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-6f798ee5-ace7-4e74-877e-ec6b54f2c0e6 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] 10.222.0.16 "GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b" status: 200 len: 1470 microversion: 2.1 time: 0.160138 Oct 30 00:59:54.953317 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 227/440] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:54 2022] GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b => generated 1470 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:55.310778 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-658b454d-5a0a-4ecd-b3f3-bd46ceffad84 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:55.311545 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-658b454d-5a0a-4ecd-b3f3-bd46ceffad84 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 3f2f7c97-5868-4aab-af09-dcdd578f5aed] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:55.320437 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-658b454d-5a0a-4ecd-b3f3-bd46ceffad84 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:55.321067 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-658b454d-5a0a-4ecd-b3f3-bd46ceffad84 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:55.321632 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-658b454d-5a0a-4ecd-b3f3-bd46ceffad84 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:55.322600 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-658b454d-5a0a-4ecd-b3f3-bd46ceffad84 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:55.323178 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-658b454d-5a0a-4ecd-b3f3-bd46ceffad84 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:55.323742 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-658b454d-5a0a-4ecd-b3f3-bd46ceffad84 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:55.406749 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-530bd3c9-7ce5-4092-a6c7-0068aad05af9 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:55.407352 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-530bd3c9-7ce5-4092-a6c7-0068aad05af9 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:55.413890 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-530bd3c9-7ce5-4092-a6c7-0068aad05af9 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:55.414430 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-530bd3c9-7ce5-4092-a6c7-0068aad05af9 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:55.414916 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-530bd3c9-7ce5-4092-a6c7-0068aad05af9 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:55.415686 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-530bd3c9-7ce5-4092-a6c7-0068aad05af9 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:55.416165 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-530bd3c9-7ce5-4092-a6c7-0068aad05af9 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:55.416609 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-530bd3c9-7ce5-4092-a6c7-0068aad05af9 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:55.474085 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-658b454d-5a0a-4ecd-b3f3-bd46ceffad84 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:55.482346 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-658b454d-5a0a-4ecd-b3f3-bd46ceffad84 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed" status: 200 len: 1314 microversion: 2.1 time: 0.177232 Oct 30 00:59:55.482940 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 214/441] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:55 2022] GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed => generated 1314 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:55.556742 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-530bd3c9-7ce5-4092-a6c7-0068aad05af9 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:55.565116 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-530bd3c9-7ce5-4092-a6c7-0068aad05af9 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1522 microversion: 2.1 time: 0.162261 Oct 30 00:59:55.565615 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 228/442] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:55 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1522 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:55.980617 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-c5e0096f-dee4-4f18-af29-3a9dbaa0d1ba tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:55.981569 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-c5e0096f-dee4-4f18-af29-3a9dbaa0d1ba tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] [instance: 0dc14c92-6364-4981-81c3-1c00e4f8187b] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:55.991650 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c5e0096f-dee4-4f18-af29-3a9dbaa0d1ba tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:55.992493 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c5e0096f-dee4-4f18-af29-3a9dbaa0d1ba tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:55.993200 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c5e0096f-dee4-4f18-af29-3a9dbaa0d1ba tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:55.994328 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c5e0096f-dee4-4f18-af29-3a9dbaa0d1ba tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:55.995276 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c5e0096f-dee4-4f18-af29-3a9dbaa0d1ba tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:55.995742 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c5e0096f-dee4-4f18-af29-3a9dbaa0d1ba tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:56.130338 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-c5e0096f-dee4-4f18-af29-3a9dbaa0d1ba tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc8294d49c654c8f9d8f7539b359df85', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27fc853acd7b4ef3803aec20a793bd06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:56.138379 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-c5e0096f-dee4-4f18-af29-3a9dbaa0d1ba tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] 10.222.0.16 "GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b" status: 200 len: 1470 microversion: 2.1 time: 0.164829 Oct 30 00:59:56.138956 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 215/443] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:55 2022] GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b => generated 1470 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:56.506790 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-50d72fa4-8fef-42c3-b288-15b4daa05d55 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:56.507388 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-50d72fa4-8fef-42c3-b288-15b4daa05d55 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 3f2f7c97-5868-4aab-af09-dcdd578f5aed] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:56.513893 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-50d72fa4-8fef-42c3-b288-15b4daa05d55 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:56.514379 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-50d72fa4-8fef-42c3-b288-15b4daa05d55 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:56.514810 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-50d72fa4-8fef-42c3-b288-15b4daa05d55 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:56.515500 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-50d72fa4-8fef-42c3-b288-15b4daa05d55 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:56.515988 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-50d72fa4-8fef-42c3-b288-15b4daa05d55 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:56.516426 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-50d72fa4-8fef-42c3-b288-15b4daa05d55 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:56.588486 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-f0409b56-7879-493b-8eda-e312940ea797 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:56.589097 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-f0409b56-7879-493b-8eda-e312940ea797 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:56.595830 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f0409b56-7879-493b-8eda-e312940ea797 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:56.596310 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f0409b56-7879-493b-8eda-e312940ea797 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:56.596755 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f0409b56-7879-493b-8eda-e312940ea797 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:56.597429 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f0409b56-7879-493b-8eda-e312940ea797 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:56.597870 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f0409b56-7879-493b-8eda-e312940ea797 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:56.598280 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f0409b56-7879-493b-8eda-e312940ea797 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:56.691683 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-50d72fa4-8fef-42c3-b288-15b4daa05d55 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:56.701607 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-50d72fa4-8fef-42c3-b288-15b4daa05d55 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed" status: 200 len: 1314 microversion: 2.1 time: 0.199211 Oct 30 00:59:56.702470 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 229/444] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:56 2022] GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed => generated 1314 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:56.759232 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-f0409b56-7879-493b-8eda-e312940ea797 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:56.767784 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-f0409b56-7879-493b-8eda-e312940ea797 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1522 microversion: 2.1 time: 0.183301 Oct 30 00:59:56.768376 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 216/445] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:56 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1522 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:57.162520 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-ffd28042-30a8-45bb-803b-8e5b87d80cd7 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:57.163152 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-ffd28042-30a8-45bb-803b-8e5b87d80cd7 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] [instance: 0dc14c92-6364-4981-81c3-1c00e4f8187b] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:57.171179 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ffd28042-30a8-45bb-803b-8e5b87d80cd7 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:57.171670 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ffd28042-30a8-45bb-803b-8e5b87d80cd7 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:57.172135 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ffd28042-30a8-45bb-803b-8e5b87d80cd7 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:57.172824 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ffd28042-30a8-45bb-803b-8e5b87d80cd7 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:57.173270 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ffd28042-30a8-45bb-803b-8e5b87d80cd7 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:57.173687 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ffd28042-30a8-45bb-803b-8e5b87d80cd7 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:57.306596 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-ffd28042-30a8-45bb-803b-8e5b87d80cd7 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc8294d49c654c8f9d8f7539b359df85', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27fc853acd7b4ef3803aec20a793bd06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:57.311906 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell1:SELECT=100 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:59:57.318697 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-ffd28042-30a8-45bb-803b-8e5b87d80cd7 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] 10.222.0.16 "GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b" status: 200 len: 1470 microversion: 2.1 time: 0.159780 Oct 30 00:59:57.319531 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 230/446] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:57 2022] GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b => generated 1470 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:57.725654 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-e24d3593-a6ef-4706-beba-b47c2e24c44d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:57.726205 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-e24d3593-a6ef-4706-beba-b47c2e24c44d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 3f2f7c97-5868-4aab-af09-dcdd578f5aed] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:57.732928 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e24d3593-a6ef-4706-beba-b47c2e24c44d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:57.733411 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e24d3593-a6ef-4706-beba-b47c2e24c44d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:57.733844 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e24d3593-a6ef-4706-beba-b47c2e24c44d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:57.734540 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e24d3593-a6ef-4706-beba-b47c2e24c44d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:57.734984 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e24d3593-a6ef-4706-beba-b47c2e24c44d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:57.735400 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e24d3593-a6ef-4706-beba-b47c2e24c44d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:57.790894 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-3b391678-7454-4bdd-a89a-305cff3d4de7 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:57.793604 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-3b391678-7454-4bdd-a89a-305cff3d4de7 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:57.800551 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3b391678-7454-4bdd-a89a-305cff3d4de7 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:57.802289 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3b391678-7454-4bdd-a89a-305cff3d4de7 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:57.803619 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3b391678-7454-4bdd-a89a-305cff3d4de7 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:57.805304 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3b391678-7454-4bdd-a89a-305cff3d4de7 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:57.805937 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3b391678-7454-4bdd-a89a-305cff3d4de7 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:57.806537 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3b391678-7454-4bdd-a89a-305cff3d4de7 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:57.927219 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-e24d3593-a6ef-4706-beba-b47c2e24c44d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:57.936464 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-e24d3593-a6ef-4706-beba-b47c2e24c44d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed" status: 200 len: 1314 microversion: 2.1 time: 0.214831 Oct 30 00:59:57.937759 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 217/447] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:57 2022] GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed => generated 1314 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:58.012196 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-3b391678-7454-4bdd-a89a-305cff3d4de7 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:58.022120 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-3b391678-7454-4bdd-a89a-305cff3d4de7 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1522 microversion: 2.1 time: 0.234977 Oct 30 00:59:58.023209 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 231/448] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:57 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1522 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:58.040334 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [req-4b3e9344-d532-4e0b-a782-04b7b7773809 req-5aa49c3e-0a65-44a0-be05-5c6587c6b2f7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0dc14c92-6364-4981-81c3-1c00e4f8187b", "name": "network-vif-deleted", "tag": "5edec0bf-c7b4-4cb7-b191-28078b9dccca"}]} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 00:59:58.056560 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-4b3e9344-d532-4e0b-a782-04b7b7773809 req-5aa49c3e-0a65-44a0-be05-5c6587c6b2f7 service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:58.057183 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-4b3e9344-d532-4e0b-a782-04b7b7773809 req-5aa49c3e-0a65-44a0-be05-5c6587c6b2f7 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:58.057719 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-4b3e9344-d532-4e0b-a782-04b7b7773809 req-5aa49c3e-0a65-44a0-be05-5c6587c6b2f7 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:58.077890 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.compute.server_external_events [req-4b3e9344-d532-4e0b-a782-04b7b7773809 req-5aa49c3e-0a65-44a0-be05-5c6587c6b2f7 service nova] Creating event network-vif-deleted:5edec0bf-c7b4-4cb7-b191-28078b9dccca for instance 0dc14c92-6364-4981-81c3-1c00e4f8187b on hv-cinder-maste Oct 30 00:59:58.083221 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [req-4b3e9344-d532-4e0b-a782-04b7b7773809 req-5aa49c3e-0a65-44a0-be05-5c6587c6b2f7 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.052185 Oct 30 00:59:58.084232 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 218/449] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 00:59:58 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 00:59:58.343179 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-5aa3ae8c-9c44-4242-bfd0-e4ed7a165bd7 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:58.343866 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-5aa3ae8c-9c44-4242-bfd0-e4ed7a165bd7 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] [instance: 0dc14c92-6364-4981-81c3-1c00e4f8187b] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:58.350589 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5aa3ae8c-9c44-4242-bfd0-e4ed7a165bd7 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:58.351085 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5aa3ae8c-9c44-4242-bfd0-e4ed7a165bd7 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:58.351522 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5aa3ae8c-9c44-4242-bfd0-e4ed7a165bd7 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:58.352282 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5aa3ae8c-9c44-4242-bfd0-e4ed7a165bd7 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:58.352735 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5aa3ae8c-9c44-4242-bfd0-e4ed7a165bd7 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:58.353155 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5aa3ae8c-9c44-4242-bfd0-e4ed7a165bd7 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:58.384561 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5aa3ae8c-9c44-4242-bfd0-e4ed7a165bd7 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:58.385040 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5aa3ae8c-9c44-4242-bfd0-e4ed7a165bd7 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:58.385508 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5aa3ae8c-9c44-4242-bfd0-e4ed7a165bd7 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:58.385960 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.objects.instance [None req-5aa3ae8c-9c44-4242-bfd0-e4ed7a165bd7 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lazy-loading 'fault' on Instance uuid 0dc14c92-6364-4981-81c3-1c00e4f8187b {{(pid=118961) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 30 00:59:58.431096 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-5aa3ae8c-9c44-4242-bfd0-e4ed7a165bd7 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc8294d49c654c8f9d8f7539b359df85', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27fc853acd7b4ef3803aec20a793bd06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:58.439465 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-5aa3ae8c-9c44-4242-bfd0-e4ed7a165bd7 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] 10.222.0.16 "GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b" status: 200 len: 1305 microversion: 2.1 time: 0.100672 Oct 30 00:59:58.440068 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 232/450] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:58 2022] GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b => generated 1305 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:58.963578 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-a13c395d-8034-4ae4-a3d0-7e1d2c01daad tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:58.964290 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-a13c395d-8034-4ae4-a3d0-7e1d2c01daad tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 3f2f7c97-5868-4aab-af09-dcdd578f5aed] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:58.970784 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a13c395d-8034-4ae4-a3d0-7e1d2c01daad tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:58.971264 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a13c395d-8034-4ae4-a3d0-7e1d2c01daad tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:58.971692 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a13c395d-8034-4ae4-a3d0-7e1d2c01daad tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:58.972388 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a13c395d-8034-4ae4-a3d0-7e1d2c01daad tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:58.972840 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a13c395d-8034-4ae4-a3d0-7e1d2c01daad tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:58.973252 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a13c395d-8034-4ae4-a3d0-7e1d2c01daad tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:59.045910 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-e87d3050-f75d-4ca8-b96b-111c82ebea06 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:59.046513 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-e87d3050-f75d-4ca8-b96b-111c82ebea06 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:59.052658 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e87d3050-f75d-4ca8-b96b-111c82ebea06 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:59.053152 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e87d3050-f75d-4ca8-b96b-111c82ebea06 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:59.053585 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e87d3050-f75d-4ca8-b96b-111c82ebea06 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:59.054467 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e87d3050-f75d-4ca8-b96b-111c82ebea06 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:59.054920 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e87d3050-f75d-4ca8-b96b-111c82ebea06 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:59.055345 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e87d3050-f75d-4ca8-b96b-111c82ebea06 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:59.147877 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-a13c395d-8034-4ae4-a3d0-7e1d2c01daad tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:59.156382 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-a13c395d-8034-4ae4-a3d0-7e1d2c01daad tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed" status: 200 len: 1314 microversion: 2.1 time: 0.197493 Oct 30 00:59:59.156926 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 219/451] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:58 2022] GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed => generated 1314 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:59.208262 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-e87d3050-f75d-4ca8-b96b-111c82ebea06 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:59.216339 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-e87d3050-f75d-4ca8-b96b-111c82ebea06 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1522 microversion: 2.1 time: 0.174455 Oct 30 00:59:59.216916 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 233/452] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:59 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1522 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:59.376775 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-4f89d941-36de-44dd-93e7-e4bcadf8f395 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:59.377326 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-4f89d941-36de-44dd-93e7-e4bcadf8f395 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: 5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:59.383574 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4f89d941-36de-44dd-93e7-e4bcadf8f395 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:59.384088 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4f89d941-36de-44dd-93e7-e4bcadf8f395 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:59.384523 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4f89d941-36de-44dd-93e7-e4bcadf8f395 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:59.385205 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4f89d941-36de-44dd-93e7-e4bcadf8f395 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:59.385649 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4f89d941-36de-44dd-93e7-e4bcadf8f395 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:59.386066 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4f89d941-36de-44dd-93e7-e4bcadf8f395 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:59.466466 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-02fab710-dd71-4409-a6c8-0d8cc747c64f tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 00:59:59.467422 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-02fab710-dd71-4409-a6c8-0d8cc747c64f tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] [instance: 0dc14c92-6364-4981-81c3-1c00e4f8187b] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 00:59:59.476703 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-02fab710-dd71-4409-a6c8-0d8cc747c64f tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:59.477395 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-02fab710-dd71-4409-a6c8-0d8cc747c64f tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:59.478041 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-02fab710-dd71-4409-a6c8-0d8cc747c64f tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:59.479097 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-02fab710-dd71-4409-a6c8-0d8cc747c64f tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:59.479802 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-02fab710-dd71-4409-a6c8-0d8cc747c64f tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:59.480450 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-02fab710-dd71-4409-a6c8-0d8cc747c64f tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:59.512133 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-02fab710-dd71-4409-a6c8-0d8cc747c64f tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 00:59:59.512626 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-02fab710-dd71-4409-a6c8-0d8cc747c64f tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 00:59:59.513049 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-02fab710-dd71-4409-a6c8-0d8cc747c64f tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 00:59:59.513486 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.objects.instance [None req-02fab710-dd71-4409-a6c8-0d8cc747c64f tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lazy-loading 'fault' on Instance uuid 0dc14c92-6364-4981-81c3-1c00e4f8187b {{(pid=118961) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 30 00:59:59.520677 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.objects.instance [None req-4f89d941-36de-44dd-93e7-e4bcadf8f395 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lazy-loading 'services' on Instance uuid 5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a {{(pid=118962) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 30 00:59:59.555184 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-02fab710-dd71-4409-a6c8-0d8cc747c64f tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc8294d49c654c8f9d8f7539b359df85', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27fc853acd7b4ef3803aec20a793bd06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 00:59:59.561323 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-02fab710-dd71-4409-a6c8-0d8cc747c64f tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] 10.222.0.16 "GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b" status: 200 len: 1305 microversion: 2.1 time: 0.101173 Oct 30 00:59:59.561914 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 234/453] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 00:59:59 2022] GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b => generated 1305 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 00:59:59.625116 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:59:59.637463 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-4f89d941-36de-44dd-93e7-e4bcadf8f395 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "DELETE /compute/v2.1/servers/5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a/os-volume_attachments/cbb8a932-08ad-4afc-a8b0-b4f53b511ecc" status: 202 len: 0 microversion: 2.1 time: 0.264808 Oct 30 00:59:59.638046 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 220/454] 10.222.0.16 () {58 vars in 1416 bytes} [Sun Oct 30 00:59:59 2022] DELETE /compute/v2.1/servers/5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a/os-volume_attachments/cbb8a932-08ad-4afc-a8b0-b4f53b511ecc => generated 0 bytes in 266 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 30 01:00:00.179014 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-1cfdcb2e-3750-47b5-8fac-252d701a9a1a tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:00.179593 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-1cfdcb2e-3750-47b5-8fac-252d701a9a1a tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 3f2f7c97-5868-4aab-af09-dcdd578f5aed] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:00.185823 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1cfdcb2e-3750-47b5-8fac-252d701a9a1a tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:00.186314 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1cfdcb2e-3750-47b5-8fac-252d701a9a1a tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:00.186773 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1cfdcb2e-3750-47b5-8fac-252d701a9a1a tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:00.187638 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1cfdcb2e-3750-47b5-8fac-252d701a9a1a tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:00.188113 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1cfdcb2e-3750-47b5-8fac-252d701a9a1a tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:00.188528 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1cfdcb2e-3750-47b5-8fac-252d701a9a1a tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:00.240729 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-5c4514cd-4fdb-4c1a-af2b-c468350c440d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:00.241506 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-5c4514cd-4fdb-4c1a-af2b-c468350c440d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:00.258451 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5c4514cd-4fdb-4c1a-af2b-c468350c440d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:00.258829 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5c4514cd-4fdb-4c1a-af2b-c468350c440d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:00.259255 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5c4514cd-4fdb-4c1a-af2b-c468350c440d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:00.259994 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5c4514cd-4fdb-4c1a-af2b-c468350c440d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:00.260439 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5c4514cd-4fdb-4c1a-af2b-c468350c440d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:00.260852 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5c4514cd-4fdb-4c1a-af2b-c468350c440d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:00.382949 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-1cfdcb2e-3750-47b5-8fac-252d701a9a1a tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:00.391013 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-1cfdcb2e-3750-47b5-8fac-252d701a9a1a tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed" status: 200 len: 1314 microversion: 2.1 time: 0.215855 Oct 30 01:00:00.391588 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 235/455] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:00 2022] GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed => generated 1314 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:00.396128 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [req-78c414a8-a41e-4ca9-9dcb-017e5fae8ca9 req-57a74898-2173-417f-921e-5f15af0aa59d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3f2f7c97-5868-4aab-af09-dcdd578f5aed", "name": "network-vif-plugged", "status": "completed", "tag": "642e374e-315b-4436-8c38-4da776b5b51d"}]} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:00:00.405356 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-78c414a8-a41e-4ca9-9dcb-017e5fae8ca9 req-57a74898-2173-417f-921e-5f15af0aa59d service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:00.405914 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-78c414a8-a41e-4ca9-9dcb-017e5fae8ca9 req-57a74898-2173-417f-921e-5f15af0aa59d service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:00.406449 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-78c414a8-a41e-4ca9-9dcb-017e5fae8ca9 req-57a74898-2173-417f-921e-5f15af0aa59d service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:00.417703 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.compute.server_external_events [req-78c414a8-a41e-4ca9-9dcb-017e5fae8ca9 req-57a74898-2173-417f-921e-5f15af0aa59d service nova] Creating event network-vif-plugged:642e374e-315b-4436-8c38-4da776b5b51d for instance 3f2f7c97-5868-4aab-af09-dcdd578f5aed on hv-cinder-maste Oct 30 01:00:00.423212 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [req-78c414a8-a41e-4ca9-9dcb-017e5fae8ca9 req-57a74898-2173-417f-921e-5f15af0aa59d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030411 Oct 30 01:00:00.424056 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 236/456] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:00:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:00:00.429936 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-5c4514cd-4fdb-4c1a-af2b-c468350c440d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:00.437951 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-5c4514cd-4fdb-4c1a-af2b-c468350c440d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1522 microversion: 2.1 time: 0.202285 Oct 30 01:00:00.438522 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 221/457] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:00 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1522 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 01:00:00.585212 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-7b6fe6a5-7545-4f2e-81d0-4f5205e111f2 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:00.585899 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-7b6fe6a5-7545-4f2e-81d0-4f5205e111f2 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] [instance: 0dc14c92-6364-4981-81c3-1c00e4f8187b] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:00.592690 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7b6fe6a5-7545-4f2e-81d0-4f5205e111f2 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:00.593103 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7b6fe6a5-7545-4f2e-81d0-4f5205e111f2 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:00.593533 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7b6fe6a5-7545-4f2e-81d0-4f5205e111f2 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:00.594505 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7b6fe6a5-7545-4f2e-81d0-4f5205e111f2 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:00.594981 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7b6fe6a5-7545-4f2e-81d0-4f5205e111f2 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:00.595395 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7b6fe6a5-7545-4f2e-81d0-4f5205e111f2 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:00.626239 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7b6fe6a5-7545-4f2e-81d0-4f5205e111f2 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:00.626703 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7b6fe6a5-7545-4f2e-81d0-4f5205e111f2 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:00.627120 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7b6fe6a5-7545-4f2e-81d0-4f5205e111f2 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:00.627554 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.objects.instance [None req-7b6fe6a5-7545-4f2e-81d0-4f5205e111f2 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lazy-loading 'fault' on Instance uuid 0dc14c92-6364-4981-81c3-1c00e4f8187b {{(pid=118961) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 30 01:00:00.668393 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-7b6fe6a5-7545-4f2e-81d0-4f5205e111f2 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc8294d49c654c8f9d8f7539b359df85', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27fc853acd7b4ef3803aec20a793bd06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:00.678542 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-7b6fe6a5-7545-4f2e-81d0-4f5205e111f2 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] 10.222.0.16 "GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b" status: 200 len: 1305 microversion: 2.1 time: 0.097338 Oct 30 01:00:00.679355 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 237/458] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:00 2022] GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b => generated 1305 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:01.418754 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-8f406c24-afa8-42eb-9bc3-75edee8b82db tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:01.419749 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-8f406c24-afa8-42eb-9bc3-75edee8b82db tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 3f2f7c97-5868-4aab-af09-dcdd578f5aed] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:01.427887 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8f406c24-afa8-42eb-9bc3-75edee8b82db tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:01.428363 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8f406c24-afa8-42eb-9bc3-75edee8b82db tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:01.428772 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8f406c24-afa8-42eb-9bc3-75edee8b82db tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:01.429446 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8f406c24-afa8-42eb-9bc3-75edee8b82db tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:01.429887 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8f406c24-afa8-42eb-9bc3-75edee8b82db tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:01.430294 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8f406c24-afa8-42eb-9bc3-75edee8b82db tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:01.462945 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-ddbb42e3-d827-4248-9f1a-f64f96a21e74 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:01.463526 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-ddbb42e3-d827-4248-9f1a-f64f96a21e74 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:01.472695 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ddbb42e3-d827-4248-9f1a-f64f96a21e74 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:01.473308 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ddbb42e3-d827-4248-9f1a-f64f96a21e74 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:01.473888 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ddbb42e3-d827-4248-9f1a-f64f96a21e74 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:01.474874 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ddbb42e3-d827-4248-9f1a-f64f96a21e74 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:01.475516 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ddbb42e3-d827-4248-9f1a-f64f96a21e74 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:01.476220 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ddbb42e3-d827-4248-9f1a-f64f96a21e74 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:01.632219 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-8f406c24-afa8-42eb-9bc3-75edee8b82db tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:01.640383 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-8f406c24-afa8-42eb-9bc3-75edee8b82db tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed" status: 200 len: 1314 microversion: 2.1 time: 0.228306 Oct 30 01:00:01.641576 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 222/459] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:01 2022] GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed => generated 1314 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:01.660865 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-ddbb42e3-d827-4248-9f1a-f64f96a21e74 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:01.668567 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-ddbb42e3-d827-4248-9f1a-f64f96a21e74 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1522 microversion: 2.1 time: 0.210584 Oct 30 01:00:01.669153 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 238/460] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:01 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1522 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:01.701301 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-b6632ab6-8a43-427b-b6e3-e4b5d922fd80 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:01.701914 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-b6632ab6-8a43-427b-b6e3-e4b5d922fd80 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] [instance: 0dc14c92-6364-4981-81c3-1c00e4f8187b] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:01.708150 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b6632ab6-8a43-427b-b6e3-e4b5d922fd80 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:01.708617 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b6632ab6-8a43-427b-b6e3-e4b5d922fd80 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:01.709033 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b6632ab6-8a43-427b-b6e3-e4b5d922fd80 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:01.709716 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b6632ab6-8a43-427b-b6e3-e4b5d922fd80 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:01.710158 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b6632ab6-8a43-427b-b6e3-e4b5d922fd80 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:01.710566 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b6632ab6-8a43-427b-b6e3-e4b5d922fd80 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:01.723707 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.wsgi [None req-b6632ab6-8a43-427b-b6e3-e4b5d922fd80 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] HTTP exception thrown: Instance 0dc14c92-6364-4981-81c3-1c00e4f8187b could not be found. Oct 30 01:00:01.725389 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-b6632ab6-8a43-427b-b6e3-e4b5d922fd80 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Request method failure captured: Oct 30 01:00:01.725389 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: request: GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b HTTP/1.1 Oct 30 01:00:01.725389 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept: application/json Oct 30 01:00:01.725389 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept-Encoding: identity Oct 30 01:00:01.725389 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Connection: close Oct 30 01:00:01.725389 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Length: 0 Oct 30 01:00:01.725389 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Type: application/json Oct 30 01:00:01.725389 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Host: 10.222.0.16 Oct 30 01:00:01.725389 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Openstack-System-Scope: None Oct 30 01:00:01.725389 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: User-Agent: python-urllib3/1.26.12 Oct 30 01:00:01.725389 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Auth-Token: gAAAAABjXcxokoZbKkVGYuR2TE6SKUkAovd5jwx7VhAAE-VESiJrLOOD2z3PEjNwfIV0G3ywHRuFIJqvtwlU0HdlQNZNjBtKQRSR8Q9pmYYNrPLe0B3GUT-ncluu8B87Hsb9kGJRwBlBDZ0OBW7LKPRpBke-1hywkTRm25rcTTdiCV_4JZjZ60c Oct 30 01:00:01.725389 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Id: None Oct 30 01:00:01.725389 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Name: None Oct 30 01:00:01.725389 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Identity-Status: Confirmed Oct 30 01:00:01.725389 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Is-Admin-Project: True Oct 30 01:00:01.725389 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Id: default Oct 30 01:00:01.725389 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Name: Default Oct 30 01:00:01.725389 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Id: 27fc853acd7b4ef3803aec20a793bd06 Oct 30 01:00:01.725389 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Name: tempest-VolumesNegativeTest-548011193 Oct 30 01:00:01.725389 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Role: reader,member Oct 30 01:00:01.725389 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Roles: reader,member Oct 30 01:00:01.726939 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_27fc853acd7b4ef3803aec20a793bd06"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/27fc853acd7b4ef3803aec20a793bd06"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/27fc853acd7b4ef3803aec20a793bd06"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/27fc853acd7b4ef3803aec20a793bd06"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 01:00:01.726939 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant: tempest-VolumesNegativeTest-548011193 Oct 30 01:00:01.726939 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Id: 27fc853acd7b4ef3803aec20a793bd06 Oct 30 01:00:01.726939 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Name: tempest-VolumesNegativeTest-548011193 Oct 30 01:00:01.726939 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User: tempest-VolumesNegativeTest-548011193-project Oct 30 01:00:01.726939 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Id: default Oct 30 01:00:01.726939 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Name: Default Oct 30 01:00:01.726939 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Id: bc8294d49c654c8f9d8f7539b359df85 Oct 30 01:00:01.726939 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Name: tempest-VolumesNegativeTest-548011193-project Oct 30 01:00:01.726939 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: method: > Oct 30 01:00:01.726939 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: exception: Instance 0dc14c92-6364-4981-81c3-1c00e4f8187b could not be found. Oct 30 01:00:01.726939 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: action_args: {'id': '0dc14c92-6364-4981-81c3-1c00e4f8187b'} Oct 30 01:00:01.728103 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 01:00:01.728103 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:00:01.728103 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 30 01:00:01.728103 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 30 01:00:01.728103 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 30 01:00:01.728103 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 30 01:00:01.728103 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 30 01:00:01.728103 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 30 01:00:01.728103 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 30 01:00:01.728103 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 30 01:00:01.728103 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0dc14c92-6364-4981-81c3-1c00e4f8187b could not be found. Oct 30 01:00:01.728103 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:00:01.728103 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:00:01.728103 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:00:01.728103 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:00:01.728103 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:00:01.728103 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:00:01.728103 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 01:00:01.728103 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 01:00:01.728103 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 01:00:01.729510 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 01:00:01.729510 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 30 01:00:01.729510 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 30 01:00:01.729510 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 30 01:00:01.729510 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 30 01:00:01.729510 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 30 01:00:01.729510 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 30 01:00:01.729510 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0dc14c92-6364-4981-81c3-1c00e4f8187b could not be found. Oct 30 01:00:01.729510 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:00:01.729510 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:00:01.729510 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:00:01.729510 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:00:01.729510 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:00:01.729510 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:00:01.729510 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 01:00:01.729510 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 01:00:01.729510 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0dc14c92-6364-4981-81c3-1c00e4f8187b could not be found. Oct 30 01:00:01.729510 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:00:01.730769 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-b6632ab6-8a43-427b-b6e3-e4b5d922fd80 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] Returning 404 to user: Instance 0dc14c92-6364-4981-81c3-1c00e4f8187b could not be found. {{(pid=118962) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 01:00:01.730769 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-b6632ab6-8a43-427b-b6e3-e4b5d922fd80 tempest-VolumesNegativeTest-548011193 tempest-VolumesNegativeTest-548011193-project] 10.222.0.16 "GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b" status: 404 len: 111 microversion: 2.1 time: 0.029665 Oct 30 01:00:01.730769 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 223/461] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:01 2022] GET /compute/v2.1/servers/0dc14c92-6364-4981-81c3-1c00e4f8187b => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 30 01:00:02.664853 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-02ad71a9-6dac-43f0-af38-31075ef60f21 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:02.665468 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-02ad71a9-6dac-43f0-af38-31075ef60f21 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 3f2f7c97-5868-4aab-af09-dcdd578f5aed] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:02.671760 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-02ad71a9-6dac-43f0-af38-31075ef60f21 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:02.672552 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-02ad71a9-6dac-43f0-af38-31075ef60f21 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:02.672991 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-02ad71a9-6dac-43f0-af38-31075ef60f21 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:02.673687 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-02ad71a9-6dac-43f0-af38-31075ef60f21 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:02.674215 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-02ad71a9-6dac-43f0-af38-31075ef60f21 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:02.674645 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-02ad71a9-6dac-43f0-af38-31075ef60f21 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:02.691692 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-f520f9ec-2908-4e08-ad4d-41e1b0637cca tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:02.692962 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-f520f9ec-2908-4e08-ad4d-41e1b0637cca tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:02.699623 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f520f9ec-2908-4e08-ad4d-41e1b0637cca tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:02.700229 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f520f9ec-2908-4e08-ad4d-41e1b0637cca tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:02.701118 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f520f9ec-2908-4e08-ad4d-41e1b0637cca tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:02.702748 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f520f9ec-2908-4e08-ad4d-41e1b0637cca tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:02.703447 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f520f9ec-2908-4e08-ad4d-41e1b0637cca tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:02.703911 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f520f9ec-2908-4e08-ad4d-41e1b0637cca tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:02.871921 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-02ad71a9-6dac-43f0-af38-31075ef60f21 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:02.882727 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-02ad71a9-6dac-43f0-af38-31075ef60f21 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed" status: 200 len: 1314 microversion: 2.1 time: 0.221712 Oct 30 01:00:02.895496 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 239/462] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:02 2022] GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed => generated 1314 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:02.926686 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-f520f9ec-2908-4e08-ad4d-41e1b0637cca tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:02.934788 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-f520f9ec-2908-4e08-ad4d-41e1b0637cca tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1522 microversion: 2.1 time: 0.247010 Oct 30 01:00:02.935345 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 224/463] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:02 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1522 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:03.906356 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-c12c3aa1-581b-4877-97d3-ddac5962de3b tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:03.906966 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-c12c3aa1-581b-4877-97d3-ddac5962de3b tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 3f2f7c97-5868-4aab-af09-dcdd578f5aed] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:03.913633 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c12c3aa1-581b-4877-97d3-ddac5962de3b tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:03.914097 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c12c3aa1-581b-4877-97d3-ddac5962de3b tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:03.914523 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c12c3aa1-581b-4877-97d3-ddac5962de3b tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:03.915269 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c12c3aa1-581b-4877-97d3-ddac5962de3b tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:03.915859 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c12c3aa1-581b-4877-97d3-ddac5962de3b tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:03.916280 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c12c3aa1-581b-4877-97d3-ddac5962de3b tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:03.957782 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-6e4c3195-391f-4c26-a275-eb692d764af0 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:03.958912 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-6e4c3195-391f-4c26-a275-eb692d764af0 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:03.965719 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6e4c3195-391f-4c26-a275-eb692d764af0 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:03.966461 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6e4c3195-391f-4c26-a275-eb692d764af0 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:03.967060 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6e4c3195-391f-4c26-a275-eb692d764af0 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:03.968592 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6e4c3195-391f-4c26-a275-eb692d764af0 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:03.969435 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6e4c3195-391f-4c26-a275-eb692d764af0 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:03.970201 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6e4c3195-391f-4c26-a275-eb692d764af0 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:04.138937 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-c12c3aa1-581b-4877-97d3-ddac5962de3b tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:04.148421 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-c12c3aa1-581b-4877-97d3-ddac5962de3b tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed" status: 200 len: 1314 microversion: 2.1 time: 0.245891 Oct 30 01:00:04.149056 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 240/464] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:03 2022] GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed => generated 1314 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:04.154225 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-45028184-fa23-4b8e-8593-70af58b156be tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:04.154717 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-45028184-fa23-4b8e-8593-70af58b156be tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: 5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:04.161252 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-45028184-fa23-4b8e-8593-70af58b156be tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:04.161731 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-45028184-fa23-4b8e-8593-70af58b156be tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:04.162153 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-45028184-fa23-4b8e-8593-70af58b156be tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:04.162828 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-45028184-fa23-4b8e-8593-70af58b156be tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:04.163434 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-45028184-fa23-4b8e-8593-70af58b156be tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:04.163896 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-45028184-fa23-4b8e-8593-70af58b156be tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:04.172932 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-6e4c3195-391f-4c26-a275-eb692d764af0 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:04.181333 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-6e4c3195-391f-4c26-a275-eb692d764af0 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1522 microversion: 2.1 time: 0.227247 Oct 30 01:00:04.181902 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 225/465] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:03 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1522 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 01:00:04.199514 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-45028184-fa23-4b8e-8593-70af58b156be tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: 5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a] Going to try to terminate instance {{(pid=118961) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 30 01:00:04.278415 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-45028184-fa23-4b8e-8593-70af58b156be tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "DELETE /compute/v2.1/servers/5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a" status: 204 len: 0 microversion: 2.1 time: 0.127951 Oct 30 01:00:04.279001 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 241/466] 10.222.0.16 () {58 vars in 1239 bytes} [Sun Oct 30 01:00:04 2022] DELETE /compute/v2.1/servers/5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a => generated 0 bytes in 129 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 30 01:00:04.287858 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-d8c60bfc-dbef-4812-8d66-8b10dbf7cd28 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:04.288483 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-d8c60bfc-dbef-4812-8d66-8b10dbf7cd28 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: 5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:04.294594 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d8c60bfc-dbef-4812-8d66-8b10dbf7cd28 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:04.295120 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d8c60bfc-dbef-4812-8d66-8b10dbf7cd28 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:04.295546 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d8c60bfc-dbef-4812-8d66-8b10dbf7cd28 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:04.296270 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d8c60bfc-dbef-4812-8d66-8b10dbf7cd28 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:04.296712 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d8c60bfc-dbef-4812-8d66-8b10dbf7cd28 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:04.297157 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d8c60bfc-dbef-4812-8d66-8b10dbf7cd28 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:04.438378 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-d8c60bfc-dbef-4812-8d66-8b10dbf7cd28 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:04.446784 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-d8c60bfc-dbef-4812-8d66-8b10dbf7cd28 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a" status: 200 len: 1514 microversion: 2.1 time: 0.162929 Oct 30 01:00:04.448203 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 226/467] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:04 2022] GET /compute/v2.1/servers/5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a => generated 1514 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 01:00:05.176310 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-f29bc50c-4a65-4b84-b7e9-e2b8698a622d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:05.177012 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-f29bc50c-4a65-4b84-b7e9-e2b8698a622d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 3f2f7c97-5868-4aab-af09-dcdd578f5aed] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:05.183287 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f29bc50c-4a65-4b84-b7e9-e2b8698a622d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:05.183774 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f29bc50c-4a65-4b84-b7e9-e2b8698a622d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:05.184235 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f29bc50c-4a65-4b84-b7e9-e2b8698a622d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:05.184934 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f29bc50c-4a65-4b84-b7e9-e2b8698a622d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:05.185380 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f29bc50c-4a65-4b84-b7e9-e2b8698a622d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:05.185801 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f29bc50c-4a65-4b84-b7e9-e2b8698a622d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:05.204530 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-155d6e41-4297-4d5d-80f2-64a133c82574 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:05.205149 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-155d6e41-4297-4d5d-80f2-64a133c82574 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:05.211764 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-155d6e41-4297-4d5d-80f2-64a133c82574 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:05.212276 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-155d6e41-4297-4d5d-80f2-64a133c82574 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:05.212697 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-155d6e41-4297-4d5d-80f2-64a133c82574 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:05.213384 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-155d6e41-4297-4d5d-80f2-64a133c82574 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:05.213826 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-155d6e41-4297-4d5d-80f2-64a133c82574 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:05.214239 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-155d6e41-4297-4d5d-80f2-64a133c82574 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:05.379871 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-f29bc50c-4a65-4b84-b7e9-e2b8698a622d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:05.387906 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-f29bc50c-4a65-4b84-b7e9-e2b8698a622d tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed" status: 200 len: 1497 microversion: 2.1 time: 0.216806 Oct 30 01:00:05.388499 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 242/468] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:05 2022] GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed => generated 1497 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:05.393591 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-155d6e41-4297-4d5d-80f2-64a133c82574 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:05.402083 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-155d6e41-4297-4d5d-80f2-64a133c82574 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1502 microversion: 2.1 time: 0.201837 Oct 30 01:00:05.402928 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 227/469] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:05 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1502 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:05.424334 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-f65faa30-14dc-481b-ab61-75bae89d5d7a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:05.424945 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-f65faa30-14dc-481b-ab61-75bae89d5d7a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: c7be44ac-3b42-4ace-ac56-65c117e839c6] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:05.431170 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f65faa30-14dc-481b-ab61-75bae89d5d7a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:05.431675 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f65faa30-14dc-481b-ab61-75bae89d5d7a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:05.432268 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f65faa30-14dc-481b-ab61-75bae89d5d7a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:05.432965 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f65faa30-14dc-481b-ab61-75bae89d5d7a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:05.433420 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f65faa30-14dc-481b-ab61-75bae89d5d7a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:05.433821 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f65faa30-14dc-481b-ab61-75bae89d5d7a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:05.472438 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-36e04252-d323-40fa-8d41-4d6e01bf576e tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:05.473066 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-36e04252-d323-40fa-8d41-4d6e01bf576e tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: 5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:05.479165 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-36e04252-d323-40fa-8d41-4d6e01bf576e tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:05.479698 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-36e04252-d323-40fa-8d41-4d6e01bf576e tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:05.480168 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-36e04252-d323-40fa-8d41-4d6e01bf576e tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:05.480947 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-36e04252-d323-40fa-8d41-4d6e01bf576e tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:05.481650 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-36e04252-d323-40fa-8d41-4d6e01bf576e tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:05.482077 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-36e04252-d323-40fa-8d41-4d6e01bf576e tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:05.634253 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-f65faa30-14dc-481b-ab61-75bae89d5d7a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:05.642480 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-f65faa30-14dc-481b-ab61-75bae89d5d7a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6" status: 200 len: 1516 microversion: 2.1 time: 0.221900 Oct 30 01:00:05.657337 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 243/470] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:05 2022] GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6 => generated 1516 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:05.661955 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-36e04252-d323-40fa-8d41-4d6e01bf576e tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:05.670395 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-36e04252-d323-40fa-8d41-4d6e01bf576e tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a" status: 200 len: 1514 microversion: 2.1 time: 0.201972 Oct 30 01:00:05.671550 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 228/471] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:05 2022] GET /compute/v2.1/servers/5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a => generated 1514 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:06.053461 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-33d05def-9e2e-4165-a740-a8cda1303594 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:06.054220 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-33d05def-9e2e-4165-a740-a8cda1303594 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 3f2f7c97-5868-4aab-af09-dcdd578f5aed] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:06.060094 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-33d05def-9e2e-4165-a740-a8cda1303594 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:06.060796 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-33d05def-9e2e-4165-a740-a8cda1303594 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:06.061231 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-33d05def-9e2e-4165-a740-a8cda1303594 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:06.061922 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-33d05def-9e2e-4165-a740-a8cda1303594 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:06.062354 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-33d05def-9e2e-4165-a740-a8cda1303594 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:06.062808 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-33d05def-9e2e-4165-a740-a8cda1303594 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:06.088598 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-33d05def-9e2e-4165-a740-a8cda1303594 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 3f2f7c97-5868-4aab-af09-dcdd578f5aed] Going to try to terminate instance {{(pid=118961) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 30 01:00:06.115809 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:00:06.179523 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-33d05def-9e2e-4165-a740-a8cda1303594 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "DELETE /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed" status: 204 len: 0 microversion: 2.1 time: 0.129713 Oct 30 01:00:06.180224 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 244/472] 10.222.0.16 () {58 vars in 1239 bytes} [Sun Oct 30 01:00:06 2022] DELETE /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed => generated 0 bytes in 131 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 30 01:00:06.188702 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-1bd4d972-a81c-4599-a961-2363daf6fb6c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:06.189473 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-1bd4d972-a81c-4599-a961-2363daf6fb6c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 3f2f7c97-5868-4aab-af09-dcdd578f5aed] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:06.195337 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1bd4d972-a81c-4599-a961-2363daf6fb6c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:06.195838 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1bd4d972-a81c-4599-a961-2363daf6fb6c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:06.196249 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1bd4d972-a81c-4599-a961-2363daf6fb6c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:06.197143 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1bd4d972-a81c-4599-a961-2363daf6fb6c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:06.197577 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1bd4d972-a81c-4599-a961-2363daf6fb6c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:06.197995 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1bd4d972-a81c-4599-a961-2363daf6fb6c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:06.358462 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-1bd4d972-a81c-4599-a961-2363daf6fb6c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:06.366302 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-1bd4d972-a81c-4599-a961-2363daf6fb6c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed" status: 200 len: 1503 microversion: 2.1 time: 0.181441 Oct 30 01:00:06.366794 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 229/473] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:06 2022] GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed => generated 1503 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 01:00:06.695364 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-860af22f-5487-4b7d-ac86-b47115a793ce tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:06.696001 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-860af22f-5487-4b7d-ac86-b47115a793ce tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: 5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:06.702504 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-860af22f-5487-4b7d-ac86-b47115a793ce tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:06.702985 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-860af22f-5487-4b7d-ac86-b47115a793ce tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:06.703581 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-860af22f-5487-4b7d-ac86-b47115a793ce tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:06.704410 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-860af22f-5487-4b7d-ac86-b47115a793ce tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:06.704851 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-860af22f-5487-4b7d-ac86-b47115a793ce tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:06.705287 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-860af22f-5487-4b7d-ac86-b47115a793ce tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:06.847746 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-860af22f-5487-4b7d-ac86-b47115a793ce tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:06.856213 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-860af22f-5487-4b7d-ac86-b47115a793ce tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a" status: 200 len: 1514 microversion: 2.1 time: 0.165212 Oct 30 01:00:06.856804 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 245/474] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:06 2022] GET /compute/v2.1/servers/5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a => generated 1514 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:07.390808 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-93fedd75-77d9-4194-9c9c-664e9805e34c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:07.391413 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-93fedd75-77d9-4194-9c9c-664e9805e34c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 3f2f7c97-5868-4aab-af09-dcdd578f5aed] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:07.398319 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-93fedd75-77d9-4194-9c9c-664e9805e34c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:07.398832 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-93fedd75-77d9-4194-9c9c-664e9805e34c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:07.399294 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-93fedd75-77d9-4194-9c9c-664e9805e34c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:07.400045 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-93fedd75-77d9-4194-9c9c-664e9805e34c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:07.400489 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-93fedd75-77d9-4194-9c9c-664e9805e34c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:07.400906 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-93fedd75-77d9-4194-9c9c-664e9805e34c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:07.540626 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-93fedd75-77d9-4194-9c9c-664e9805e34c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:07.548728 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-93fedd75-77d9-4194-9c9c-664e9805e34c tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed" status: 200 len: 1503 microversion: 2.1 time: 0.161859 Oct 30 01:00:07.549300 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 230/475] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:07 2022] GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed => generated 1503 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:07.883638 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-44530a21-03a6-4b8e-bc88-cb7332a80cfa tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:07.884484 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-44530a21-03a6-4b8e-bc88-cb7332a80cfa tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: 5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:07.892641 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-44530a21-03a6-4b8e-bc88-cb7332a80cfa tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:07.893276 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-44530a21-03a6-4b8e-bc88-cb7332a80cfa tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:07.893846 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-44530a21-03a6-4b8e-bc88-cb7332a80cfa tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:07.894819 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-44530a21-03a6-4b8e-bc88-cb7332a80cfa tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:07.895414 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-44530a21-03a6-4b8e-bc88-cb7332a80cfa tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:07.896008 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-44530a21-03a6-4b8e-bc88-cb7332a80cfa tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:08.033104 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-44530a21-03a6-4b8e-bc88-cb7332a80cfa tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:08.041893 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-44530a21-03a6-4b8e-bc88-cb7332a80cfa tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a" status: 200 len: 1514 microversion: 2.1 time: 0.163754 Oct 30 01:00:08.042461 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 246/476] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:07 2022] GET /compute/v2.1/servers/5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a => generated 1514 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:08.154098 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-787d6290-1de2-40ae-95ee-89ff229c34e8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:00:08.156468 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-787d6290-1de2-40ae-95ee-89ff229c34e8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: c7be44ac-3b42-4ace-ac56-65c117e839c6] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:08.162832 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-787d6290-1de2-40ae-95ee-89ff229c34e8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:08.163311 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-787d6290-1de2-40ae-95ee-89ff229c34e8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:08.163758 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-787d6290-1de2-40ae-95ee-89ff229c34e8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:08.164463 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-787d6290-1de2-40ae-95ee-89ff229c34e8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:08.164910 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-787d6290-1de2-40ae-95ee-89ff229c34e8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:08.165326 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-787d6290-1de2-40ae-95ee-89ff229c34e8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:08.187948 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.objects.instance [None req-787d6290-1de2-40ae-95ee-89ff229c34e8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lazy-loading 'flavor' on Instance uuid c7be44ac-3b42-4ace-ac56-65c117e839c6 {{(pid=118962) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 30 01:00:08.272845 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-787d6290-1de2-40ae-95ee-89ff229c34e8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "POST /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6/action" status: 200 len: 32 microversion: 2.1 time: 0.122828 Oct 30 01:00:08.273469 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 231/477] 10.222.0.16 () {60 vars in 1278 bytes} [Sun Oct 30 01:00:08 2022] POST /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6/action => generated 32 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 30 01:00:08.283350 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-e86c8978-69fe-4e83-bbaf-597095c57989 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:08.283980 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-e86c8978-69fe-4e83-bbaf-597095c57989 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: c7be44ac-3b42-4ace-ac56-65c117e839c6] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:08.290623 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e86c8978-69fe-4e83-bbaf-597095c57989 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:08.291119 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e86c8978-69fe-4e83-bbaf-597095c57989 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:08.291552 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e86c8978-69fe-4e83-bbaf-597095c57989 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:08.292324 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e86c8978-69fe-4e83-bbaf-597095c57989 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:08.292761 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e86c8978-69fe-4e83-bbaf-597095c57989 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:08.293216 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e86c8978-69fe-4e83-bbaf-597095c57989 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:08.430376 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-e86c8978-69fe-4e83-bbaf-597095c57989 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:08.441718 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-e86c8978-69fe-4e83-bbaf-597095c57989 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6" status: 200 len: 1522 microversion: 2.1 time: 0.162400 Oct 30 01:00:08.442268 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 247/478] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:08 2022] GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6 => generated 1522 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:08.578843 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-351a6a34-35fd-45ed-8086-c8580b2bd5cc tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:08.579920 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-351a6a34-35fd-45ed-8086-c8580b2bd5cc tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 3f2f7c97-5868-4aab-af09-dcdd578f5aed] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:08.590569 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-351a6a34-35fd-45ed-8086-c8580b2bd5cc tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:08.591361 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-351a6a34-35fd-45ed-8086-c8580b2bd5cc tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:08.592152 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-351a6a34-35fd-45ed-8086-c8580b2bd5cc tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:08.593391 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-351a6a34-35fd-45ed-8086-c8580b2bd5cc tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:08.594096 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-351a6a34-35fd-45ed-8086-c8580b2bd5cc tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:08.594825 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-351a6a34-35fd-45ed-8086-c8580b2bd5cc tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:08.745654 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-351a6a34-35fd-45ed-8086-c8580b2bd5cc tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:08.753558 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-351a6a34-35fd-45ed-8086-c8580b2bd5cc tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed" status: 200 len: 1503 microversion: 2.1 time: 0.182956 Oct 30 01:00:08.754221 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 232/479] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:08 2022] GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed => generated 1503 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:09.069740 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-c80f384d-df69-475d-b4ce-f79ed7e4f7c5 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:09.070698 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-c80f384d-df69-475d-b4ce-f79ed7e4f7c5 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: 5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:09.081409 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c80f384d-df69-475d-b4ce-f79ed7e4f7c5 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:09.082093 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c80f384d-df69-475d-b4ce-f79ed7e4f7c5 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:09.082548 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c80f384d-df69-475d-b4ce-f79ed7e4f7c5 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:09.083253 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c80f384d-df69-475d-b4ce-f79ed7e4f7c5 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:09.083701 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c80f384d-df69-475d-b4ce-f79ed7e4f7c5 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:09.084171 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c80f384d-df69-475d-b4ce-f79ed7e4f7c5 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:09.226543 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-c80f384d-df69-475d-b4ce-f79ed7e4f7c5 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:09.234721 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-c80f384d-df69-475d-b4ce-f79ed7e4f7c5 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a" status: 200 len: 1514 microversion: 2.1 time: 0.171613 Oct 30 01:00:09.235222 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 248/480] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:09 2022] GET /compute/v2.1/servers/5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a => generated 1514 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:09.468177 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-1756a3c2-91dc-41da-a58a-85fe5f818e46 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:09.469114 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-1756a3c2-91dc-41da-a58a-85fe5f818e46 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: c7be44ac-3b42-4ace-ac56-65c117e839c6] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:09.476027 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1756a3c2-91dc-41da-a58a-85fe5f818e46 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:09.476513 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1756a3c2-91dc-41da-a58a-85fe5f818e46 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:09.476942 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1756a3c2-91dc-41da-a58a-85fe5f818e46 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:09.477631 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1756a3c2-91dc-41da-a58a-85fe5f818e46 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:09.478076 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1756a3c2-91dc-41da-a58a-85fe5f818e46 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:09.479116 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1756a3c2-91dc-41da-a58a-85fe5f818e46 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:09.632647 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-1756a3c2-91dc-41da-a58a-85fe5f818e46 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:09.640378 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-1756a3c2-91dc-41da-a58a-85fe5f818e46 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6" status: 200 len: 1522 microversion: 2.1 time: 0.178506 Oct 30 01:00:09.640934 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 233/481] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:09 2022] GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6 => generated 1522 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:09.778334 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-733cc2a1-6fdb-4884-bf6c-4a78b6a1ca84 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:09.779101 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-733cc2a1-6fdb-4884-bf6c-4a78b6a1ca84 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 3f2f7c97-5868-4aab-af09-dcdd578f5aed] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:09.785665 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-733cc2a1-6fdb-4884-bf6c-4a78b6a1ca84 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:09.786325 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-733cc2a1-6fdb-4884-bf6c-4a78b6a1ca84 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:09.787071 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-733cc2a1-6fdb-4884-bf6c-4a78b6a1ca84 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:09.787907 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-733cc2a1-6fdb-4884-bf6c-4a78b6a1ca84 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:09.788423 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-733cc2a1-6fdb-4884-bf6c-4a78b6a1ca84 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:09.788989 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-733cc2a1-6fdb-4884-bf6c-4a78b6a1ca84 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:09.847651 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [req-3d684ea1-cdac-43f7-8fce-35d69c0e089d req-6b651d90-6a85-4072-a4b5-9a03a8b6cefc service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a", "name": "network-vif-deleted", "tag": "762c8731-46b3-44de-8232-dda35195aa9c"}]} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:00:09.858125 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-3d684ea1-cdac-43f7-8fce-35d69c0e089d req-6b651d90-6a85-4072-a4b5-9a03a8b6cefc service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:09.858585 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-3d684ea1-cdac-43f7-8fce-35d69c0e089d req-6b651d90-6a85-4072-a4b5-9a03a8b6cefc service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:09.858996 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-3d684ea1-cdac-43f7-8fce-35d69c0e089d req-6b651d90-6a85-4072-a4b5-9a03a8b6cefc service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:09.872088 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.compute.server_external_events [req-3d684ea1-cdac-43f7-8fce-35d69c0e089d req-6b651d90-6a85-4072-a4b5-9a03a8b6cefc service nova] Creating event network-vif-deleted:762c8731-46b3-44de-8232-dda35195aa9c for instance 5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a on hv-cinder-maste Oct 30 01:00:09.877960 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [req-3d684ea1-cdac-43f7-8fce-35d69c0e089d req-6b651d90-6a85-4072-a4b5-9a03a8b6cefc service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035566 Oct 30 01:00:09.881974 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 234/482] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:00:09 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:00:09.933525 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-733cc2a1-6fdb-4884-bf6c-4a78b6a1ca84 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:09.941268 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-733cc2a1-6fdb-4884-bf6c-4a78b6a1ca84 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed" status: 200 len: 1503 microversion: 2.1 time: 0.167491 Oct 30 01:00:09.941937 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 249/483] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:09 2022] GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed => generated 1503 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:10.263398 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-465fb3f2-c0fa-471c-b025-6fdfb4d9122d tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:10.264132 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-465fb3f2-c0fa-471c-b025-6fdfb4d9122d tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: 5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:10.270972 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-465fb3f2-c0fa-471c-b025-6fdfb4d9122d tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:10.271508 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-465fb3f2-c0fa-471c-b025-6fdfb4d9122d tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:10.272099 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-465fb3f2-c0fa-471c-b025-6fdfb4d9122d tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:10.272821 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-465fb3f2-c0fa-471c-b025-6fdfb4d9122d tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:10.273602 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-465fb3f2-c0fa-471c-b025-6fdfb4d9122d tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:10.274171 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-465fb3f2-c0fa-471c-b025-6fdfb4d9122d tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:10.307256 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-465fb3f2-c0fa-471c-b025-6fdfb4d9122d tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:10.307950 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-465fb3f2-c0fa-471c-b025-6fdfb4d9122d tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:10.308576 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-465fb3f2-c0fa-471c-b025-6fdfb4d9122d tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:10.309399 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.objects.instance [None req-465fb3f2-c0fa-471c-b025-6fdfb4d9122d tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lazy-loading 'fault' on Instance uuid 5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a {{(pid=118962) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 30 01:00:10.360987 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-465fb3f2-c0fa-471c-b025-6fdfb4d9122d tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:10.370321 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-465fb3f2-c0fa-471c-b025-6fdfb4d9122d tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a" status: 200 len: 1309 microversion: 2.1 time: 0.110971 Oct 30 01:00:10.383875 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 235/484] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:10 2022] GET /compute/v2.1/servers/5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a => generated 1309 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:10.387260 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-7c02b824-6c03-43d2-b89e-98b9d78c068d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:10.387260 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-7c02b824-6c03-43d2-b89e-98b9d78c068d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:10.392795 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7c02b824-6c03-43d2-b89e-98b9d78c068d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:10.393538 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7c02b824-6c03-43d2-b89e-98b9d78c068d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:10.394112 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7c02b824-6c03-43d2-b89e-98b9d78c068d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:10.395009 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7c02b824-6c03-43d2-b89e-98b9d78c068d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:10.395449 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7c02b824-6c03-43d2-b89e-98b9d78c068d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:10.396049 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7c02b824-6c03-43d2-b89e-98b9d78c068d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:10.421851 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-7c02b824-6c03-43d2-b89e-98b9d78c068d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f/ips" status: 200 len: 152 microversion: 2.20 time: 0.040387 Oct 30 01:00:10.422656 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 250/485] 10.222.0.16 () {60 vars in 1289 bytes} [Sun Oct 30 01:00:10 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f/ips => generated 152 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 30 01:00:10.665596 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-3e885106-27b1-4fd8-9710-16b466e4d82f tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:10.666370 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-3e885106-27b1-4fd8-9710-16b466e4d82f tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: c7be44ac-3b42-4ace-ac56-65c117e839c6] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:10.686135 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3e885106-27b1-4fd8-9710-16b466e4d82f tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:10.691339 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3e885106-27b1-4fd8-9710-16b466e4d82f tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:10.691806 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3e885106-27b1-4fd8-9710-16b466e4d82f tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:10.692537 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3e885106-27b1-4fd8-9710-16b466e4d82f tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:10.693098 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3e885106-27b1-4fd8-9710-16b466e4d82f tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:10.694050 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3e885106-27b1-4fd8-9710-16b466e4d82f tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:10.723126 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:00:10.872830 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-3e885106-27b1-4fd8-9710-16b466e4d82f tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:10.882763 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-3e885106-27b1-4fd8-9710-16b466e4d82f tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6" status: 200 len: 1522 microversion: 2.1 time: 0.222047 Oct 30 01:00:10.883626 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 236/486] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:10 2022] GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6 => generated 1522 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:10.964880 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-d0720cd4-ecb2-4113-9992-a9735286565b tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:10.965496 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-d0720cd4-ecb2-4113-9992-a9735286565b tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 3f2f7c97-5868-4aab-af09-dcdd578f5aed] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:10.971691 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d0720cd4-ecb2-4113-9992-a9735286565b tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:10.972439 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d0720cd4-ecb2-4113-9992-a9735286565b tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:10.973032 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d0720cd4-ecb2-4113-9992-a9735286565b tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:10.973736 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d0720cd4-ecb2-4113-9992-a9735286565b tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:10.974187 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d0720cd4-ecb2-4113-9992-a9735286565b tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:10.974627 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d0720cd4-ecb2-4113-9992-a9735286565b tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:11.051595 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-d0720cd4-ecb2-4113-9992-a9735286565b tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:11.060449 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-d0720cd4-ecb2-4113-9992-a9735286565b tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed" status: 200 len: 1461 microversion: 2.1 time: 0.099460 Oct 30 01:00:11.061087 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 251/487] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:10 2022] GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed => generated 1461 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:11.395082 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-d90bcf72-58b0-419d-a8fe-ee7bb0281033 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:11.395680 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-d90bcf72-58b0-419d-a8fe-ee7bb0281033 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: 5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:11.401640 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d90bcf72-58b0-419d-a8fe-ee7bb0281033 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:11.402114 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d90bcf72-58b0-419d-a8fe-ee7bb0281033 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:11.402534 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d90bcf72-58b0-419d-a8fe-ee7bb0281033 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:11.403206 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d90bcf72-58b0-419d-a8fe-ee7bb0281033 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:11.404063 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d90bcf72-58b0-419d-a8fe-ee7bb0281033 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:11.404477 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d90bcf72-58b0-419d-a8fe-ee7bb0281033 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:11.418685 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.wsgi [None req-d90bcf72-58b0-419d-a8fe-ee7bb0281033 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] HTTP exception thrown: Instance 5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a could not be found. Oct 30 01:00:11.420659 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-d90bcf72-58b0-419d-a8fe-ee7bb0281033 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Request method failure captured: Oct 30 01:00:11.420659 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: request: GET /compute/v2.1/servers/5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a HTTP/1.1 Oct 30 01:00:11.420659 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept: application/json Oct 30 01:00:11.420659 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept-Encoding: identity Oct 30 01:00:11.420659 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Connection: close Oct 30 01:00:11.420659 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Length: 0 Oct 30 01:00:11.420659 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Type: application/json Oct 30 01:00:11.420659 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Host: 10.222.0.16 Oct 30 01:00:11.420659 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Openstack-System-Scope: None Oct 30 01:00:11.420659 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: User-Agent: python-urllib3/1.26.12 Oct 30 01:00:11.420659 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Auth-Token: gAAAAABjXcwpEK4P3cM0ywCLZB1a-Aa4hWbP8jTJ4dGTi7fm1Gw71o6nFRneJTYcyExGVypGWFJMASQBgQOkcdgBwyyFxBmsEpyudU9H41QTjw7SzfiyEzKMQ4OwemWWuMPbeXo3pfyEMR9xuKPgDSRAEQCefZCab_ZKaD658JjLDxFMQOGtrcs Oct 30 01:00:11.420659 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Id: None Oct 30 01:00:11.420659 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Name: None Oct 30 01:00:11.420659 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Identity-Status: Confirmed Oct 30 01:00:11.420659 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Is-Admin-Project: True Oct 30 01:00:11.420659 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Id: default Oct 30 01:00:11.420659 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Name: Default Oct 30 01:00:11.420659 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Id: 2fbc73a853584aaba6109090bd6b1281 Oct 30 01:00:11.420659 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Name: tempest-VolumesSnapshotTestJSON-761222885 Oct 30 01:00:11.420659 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Role: reader,member Oct 30 01:00:11.420659 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Roles: reader,member Oct 30 01:00:11.421954 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_2fbc73a853584aaba6109090bd6b1281"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2fbc73a853584aaba6109090bd6b1281"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/2fbc73a853584aaba6109090bd6b1281"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2fbc73a853584aaba6109090bd6b1281"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 01:00:11.421954 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant: tempest-VolumesSnapshotTestJSON-761222885 Oct 30 01:00:11.421954 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Id: 2fbc73a853584aaba6109090bd6b1281 Oct 30 01:00:11.421954 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-761222885 Oct 30 01:00:11.421954 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User: tempest-VolumesSnapshotTestJSON-761222885-project Oct 30 01:00:11.421954 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Id: default Oct 30 01:00:11.421954 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Name: Default Oct 30 01:00:11.421954 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Id: ddeaa7a90ee7483590e6bc0d189e1e2a Oct 30 01:00:11.421954 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Name: tempest-VolumesSnapshotTestJSON-761222885-project Oct 30 01:00:11.421954 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: method: > Oct 30 01:00:11.421954 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: exception: Instance 5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a could not be found. Oct 30 01:00:11.421954 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: action_args: {'id': '5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a'} Oct 30 01:00:11.423098 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 01:00:11.423098 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:00:11.423098 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 30 01:00:11.423098 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 30 01:00:11.423098 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 30 01:00:11.423098 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 30 01:00:11.423098 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 30 01:00:11.423098 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 30 01:00:11.423098 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 30 01:00:11.423098 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 30 01:00:11.423098 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a could not be found. Oct 30 01:00:11.423098 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:00:11.423098 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:00:11.423098 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:00:11.423098 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:00:11.423098 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:00:11.423098 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:00:11.423098 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 01:00:11.423098 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 01:00:11.423098 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 01:00:11.424654 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 01:00:11.424654 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 30 01:00:11.424654 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 30 01:00:11.424654 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 30 01:00:11.424654 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 30 01:00:11.424654 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 30 01:00:11.424654 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 30 01:00:11.424654 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a could not be found. Oct 30 01:00:11.424654 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:00:11.424654 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:00:11.424654 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:00:11.424654 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:00:11.424654 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:00:11.424654 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:00:11.424654 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 01:00:11.424654 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 01:00:11.424654 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a could not be found. Oct 30 01:00:11.424654 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:00:11.426541 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-d90bcf72-58b0-419d-a8fe-ee7bb0281033 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Returning 404 to user: Instance 5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a could not be found. {{(pid=118962) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 01:00:11.426541 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-d90bcf72-58b0-419d-a8fe-ee7bb0281033 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a" status: 404 len: 111 microversion: 2.1 time: 0.030974 Oct 30 01:00:11.426541 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 237/488] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:11 2022] GET /compute/v2.1/servers/5aeb0bf1-cf74-4332-9a2e-31805e7a5d3a => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 30 01:00:11.906050 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [req-9427a336-64e1-4af9-9d23-b9270d016e49 req-e51815a3-4b8d-428d-bffa-acccb7bdf242 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3f2f7c97-5868-4aab-af09-dcdd578f5aed", "name": "network-vif-deleted", "tag": "642e374e-315b-4436-8c38-4da776b5b51d"}]} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:00:11.908531 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-6f30e10c-f992-4b13-a4ff-6ead9ac914ed tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:11.915596 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-6f30e10c-f992-4b13-a4ff-6ead9ac914ed tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: c7be44ac-3b42-4ace-ac56-65c117e839c6] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:11.919779 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-9427a336-64e1-4af9-9d23-b9270d016e49 req-e51815a3-4b8d-428d-bffa-acccb7bdf242 service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:11.920538 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-9427a336-64e1-4af9-9d23-b9270d016e49 req-e51815a3-4b8d-428d-bffa-acccb7bdf242 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:11.921047 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-9427a336-64e1-4af9-9d23-b9270d016e49 req-e51815a3-4b8d-428d-bffa-acccb7bdf242 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:11.924775 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6f30e10c-f992-4b13-a4ff-6ead9ac914ed tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:11.925563 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6f30e10c-f992-4b13-a4ff-6ead9ac914ed tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:11.925990 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6f30e10c-f992-4b13-a4ff-6ead9ac914ed tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:11.926674 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6f30e10c-f992-4b13-a4ff-6ead9ac914ed tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:11.927117 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6f30e10c-f992-4b13-a4ff-6ead9ac914ed tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:11.927525 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6f30e10c-f992-4b13-a4ff-6ead9ac914ed tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:11.934090 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.compute.server_external_events [req-9427a336-64e1-4af9-9d23-b9270d016e49 req-e51815a3-4b8d-428d-bffa-acccb7bdf242 service nova] Creating event network-vif-deleted:642e374e-315b-4436-8c38-4da776b5b51d for instance 3f2f7c97-5868-4aab-af09-dcdd578f5aed on hv-cinder-maste Oct 30 01:00:11.939093 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [req-9427a336-64e1-4af9-9d23-b9270d016e49 req-e51815a3-4b8d-428d-bffa-acccb7bdf242 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038113 Oct 30 01:00:11.939652 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 252/489] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:00:11 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:00:12.073627 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-6f30e10c-f992-4b13-a4ff-6ead9ac914ed tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:12.082866 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-6f30e10c-f992-4b13-a4ff-6ead9ac914ed tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6" status: 200 len: 1522 microversion: 2.1 time: 0.179657 Oct 30 01:00:12.083557 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 238/490] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:11 2022] GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6 => generated 1522 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 01:00:12.085245 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-742d6234-aeef-4dee-9979-4b014206b5eb tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:12.085927 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-742d6234-aeef-4dee-9979-4b014206b5eb tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 3f2f7c97-5868-4aab-af09-dcdd578f5aed] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:12.092533 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-742d6234-aeef-4dee-9979-4b014206b5eb tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:12.093289 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-742d6234-aeef-4dee-9979-4b014206b5eb tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:12.093906 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-742d6234-aeef-4dee-9979-4b014206b5eb tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:12.094748 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-742d6234-aeef-4dee-9979-4b014206b5eb tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:12.095361 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-742d6234-aeef-4dee-9979-4b014206b5eb tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:12.095802 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-742d6234-aeef-4dee-9979-4b014206b5eb tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:12.130711 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-742d6234-aeef-4dee-9979-4b014206b5eb tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:12.131194 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-742d6234-aeef-4dee-9979-4b014206b5eb tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:12.131622 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-742d6234-aeef-4dee-9979-4b014206b5eb tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:12.132094 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.objects.instance [None req-742d6234-aeef-4dee-9979-4b014206b5eb tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lazy-loading 'fault' on Instance uuid 3f2f7c97-5868-4aab-af09-dcdd578f5aed {{(pid=118961) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 30 01:00:12.176759 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-742d6234-aeef-4dee-9979-4b014206b5eb tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f198d5e999b487bad75a6b423fe1adc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dd16277cbc344c9aa8ffbc45ce1a16d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:12.183966 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-742d6234-aeef-4dee-9979-4b014206b5eb tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed" status: 200 len: 1304 microversion: 2.1 time: 0.102962 Oct 30 01:00:12.184500 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 253/491] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:12 2022] GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed => generated 1304 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 01:00:13.109997 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-fb31df98-32ed-49ef-a482-8ddfdc29def0 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:13.110966 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-fb31df98-32ed-49ef-a482-8ddfdc29def0 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: c7be44ac-3b42-4ace-ac56-65c117e839c6] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:13.121874 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-fb31df98-32ed-49ef-a482-8ddfdc29def0 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:13.122666 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-fb31df98-32ed-49ef-a482-8ddfdc29def0 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:13.123374 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-fb31df98-32ed-49ef-a482-8ddfdc29def0 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:13.124553 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-fb31df98-32ed-49ef-a482-8ddfdc29def0 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:13.125285 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-fb31df98-32ed-49ef-a482-8ddfdc29def0 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:13.125972 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-fb31df98-32ed-49ef-a482-8ddfdc29def0 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:13.208998 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-9bc1de08-3ca0-42da-90bf-ca9767ebf524 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:13.209658 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-9bc1de08-3ca0-42da-90bf-ca9767ebf524 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] [instance: 3f2f7c97-5868-4aab-af09-dcdd578f5aed] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:13.216481 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9bc1de08-3ca0-42da-90bf-ca9767ebf524 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:13.217047 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9bc1de08-3ca0-42da-90bf-ca9767ebf524 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:13.217556 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9bc1de08-3ca0-42da-90bf-ca9767ebf524 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:13.218319 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9bc1de08-3ca0-42da-90bf-ca9767ebf524 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:13.218861 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9bc1de08-3ca0-42da-90bf-ca9767ebf524 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:13.219351 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9bc1de08-3ca0-42da-90bf-ca9767ebf524 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:13.233372 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.wsgi [None req-9bc1de08-3ca0-42da-90bf-ca9767ebf524 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] HTTP exception thrown: Instance 3f2f7c97-5868-4aab-af09-dcdd578f5aed could not be found. Oct 30 01:00:13.235135 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-9bc1de08-3ca0-42da-90bf-ca9767ebf524 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Request method failure captured: Oct 30 01:00:13.235135 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: request: GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed HTTP/1.1 Oct 30 01:00:13.235135 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Accept: application/json Oct 30 01:00:13.235135 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Accept-Encoding: identity Oct 30 01:00:13.235135 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Connection: close Oct 30 01:00:13.235135 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Content-Length: 0 Oct 30 01:00:13.235135 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Content-Type: application/json Oct 30 01:00:13.235135 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Host: 10.222.0.16 Oct 30 01:00:13.235135 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Openstack-System-Scope: None Oct 30 01:00:13.235135 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: User-Agent: python-urllib3/1.26.12 Oct 30 01:00:13.235135 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Auth-Token: gAAAAABjXcxCc44LIEkKQ8a2UgGE4iirtf50DZU_H2glHE_HYBb_16OiP1wRwxnOeJW3GwZjrUbbKDl0gsmhkj628rcdOi7OiIsT6j3y617poGDGiePAXyZW0RN-KEA3m-VSK5uKK5FPKJqItmTZocqlRENpl9CFC31MgcWiu3nv_w45Yb558mg Oct 30 01:00:13.235135 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Domain-Id: None Oct 30 01:00:13.235135 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Domain-Name: None Oct 30 01:00:13.235135 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Identity-Status: Confirmed Oct 30 01:00:13.235135 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Is-Admin-Project: True Oct 30 01:00:13.235135 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Domain-Id: default Oct 30 01:00:13.235135 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Domain-Name: Default Oct 30 01:00:13.235135 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Id: dd16277cbc344c9aa8ffbc45ce1a16d2 Oct 30 01:00:13.235135 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Name: tempest-VolumesActionsTest-1212931503 Oct 30 01:00:13.235135 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Role: reader,member Oct 30 01:00:13.235135 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Roles: reader,member Oct 30 01:00:13.236459 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_dd16277cbc344c9aa8ffbc45ce1a16d2"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/dd16277cbc344c9aa8ffbc45ce1a16d2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/dd16277cbc344c9aa8ffbc45ce1a16d2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/dd16277cbc344c9aa8ffbc45ce1a16d2"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 01:00:13.236459 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant: tempest-VolumesActionsTest-1212931503 Oct 30 01:00:13.236459 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant-Id: dd16277cbc344c9aa8ffbc45ce1a16d2 Oct 30 01:00:13.236459 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant-Name: tempest-VolumesActionsTest-1212931503 Oct 30 01:00:13.236459 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User: tempest-VolumesActionsTest-1212931503-project Oct 30 01:00:13.236459 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Domain-Id: default Oct 30 01:00:13.236459 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Domain-Name: Default Oct 30 01:00:13.236459 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Id: 5f198d5e999b487bad75a6b423fe1adc Oct 30 01:00:13.236459 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Name: tempest-VolumesActionsTest-1212931503-project Oct 30 01:00:13.236459 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: method: > Oct 30 01:00:13.236459 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: exception: Instance 3f2f7c97-5868-4aab-af09-dcdd578f5aed could not be found. Oct 30 01:00:13.236459 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: action_args: {'id': '3f2f7c97-5868-4aab-af09-dcdd578f5aed'} Oct 30 01:00:13.237289 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 01:00:13.237289 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:00:13.237289 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 30 01:00:13.237289 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 30 01:00:13.237289 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 30 01:00:13.237289 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 30 01:00:13.237289 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 30 01:00:13.237289 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 30 01:00:13.237289 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 30 01:00:13.237289 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 30 01:00:13.237289 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3f2f7c97-5868-4aab-af09-dcdd578f5aed could not be found. Oct 30 01:00:13.237289 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:00:13.237289 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:00:13.237289 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:00:13.237289 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:00:13.237289 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:00:13.237289 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:00:13.237289 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 01:00:13.237289 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 01:00:13.237289 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 01:00:13.238631 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 01:00:13.238631 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 30 01:00:13.238631 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 30 01:00:13.238631 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 30 01:00:13.238631 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 30 01:00:13.238631 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 30 01:00:13.238631 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 30 01:00:13.238631 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3f2f7c97-5868-4aab-af09-dcdd578f5aed could not be found. Oct 30 01:00:13.238631 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:00:13.238631 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:00:13.238631 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:00:13.238631 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:00:13.238631 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:00:13.238631 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:00:13.238631 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 01:00:13.238631 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 01:00:13.238631 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3f2f7c97-5868-4aab-af09-dcdd578f5aed could not be found. Oct 30 01:00:13.238631 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:00:13.239885 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-9bc1de08-3ca0-42da-90bf-ca9767ebf524 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] Returning 404 to user: Instance 3f2f7c97-5868-4aab-af09-dcdd578f5aed could not be found. {{(pid=118961) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 01:00:13.239885 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-9bc1de08-3ca0-42da-90bf-ca9767ebf524 tempest-VolumesActionsTest-1212931503 tempest-VolumesActionsTest-1212931503-project] 10.222.0.16 "GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed" status: 404 len: 111 microversion: 2.1 time: 0.032485 Oct 30 01:00:13.240598 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 254/492] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:13 2022] GET /compute/v2.1/servers/3f2f7c97-5868-4aab-af09-dcdd578f5aed => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 30 01:00:13.262825 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-fb31df98-32ed-49ef-a482-8ddfdc29def0 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:13.271059 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-fb31df98-32ed-49ef-a482-8ddfdc29def0 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6" status: 200 len: 1522 microversion: 2.1 time: 0.167828 Oct 30 01:00:13.271620 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 239/493] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:13 2022] GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6 => generated 1522 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:13.643986 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-83079fb6-34a6-4122-a846-a094ebe08c69 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:00:13.644513 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-83079fb6-34a6-4122-a846-a094ebe08c69 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:13.651180 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-83079fb6-34a6-4122-a846-a094ebe08c69 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:13.651376 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-83079fb6-34a6-4122-a846-a094ebe08c69 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:13.651878 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-83079fb6-34a6-4122-a846-a094ebe08c69 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:13.652565 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-83079fb6-34a6-4122-a846-a094ebe08c69 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:13.653016 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-83079fb6-34a6-4122-a846-a094ebe08c69 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:13.653434 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-83079fb6-34a6-4122-a846-a094ebe08c69 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:13.677174 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.objects.instance [None req-83079fb6-34a6-4122-a846-a094ebe08c69 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lazy-loading 'flavor' on Instance uuid 9491a8f4-2b76-4a4e-a6bd-6720114e482f {{(pid=118961) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 30 01:00:13.931339 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-83079fb6-34a6-4122-a846-a094ebe08c69 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "POST /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f/action" status: 202 len: 0 microversion: 2.20 time: 0.291109 Oct 30 01:00:13.935454 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 255/494] 10.222.0.16 () {62 vars in 1319 bytes} [Sun Oct 30 01:00:13 2022] POST /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f/action => generated 0 bytes in 293 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 30 01:00:13.940923 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-65947b42-09cd-4319-b2ce-c03c90cddacf tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:13.941524 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-65947b42-09cd-4319-b2ce-c03c90cddacf tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:13.943435 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_api:SELECT=82 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:00:13.949104 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-65947b42-09cd-4319-b2ce-c03c90cddacf tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:13.949821 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-65947b42-09cd-4319-b2ce-c03c90cddacf tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:13.950254 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-65947b42-09cd-4319-b2ce-c03c90cddacf tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:13.951307 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-65947b42-09cd-4319-b2ce-c03c90cddacf tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:13.951945 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-65947b42-09cd-4319-b2ce-c03c90cddacf tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:13.952553 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-65947b42-09cd-4319-b2ce-c03c90cddacf tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:14.092827 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-65947b42-09cd-4319-b2ce-c03c90cddacf tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:14.101194 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-65947b42-09cd-4319-b2ce-c03c90cddacf tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:14.102637 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-65947b42-09cd-4319-b2ce-c03c90cddacf tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:14.103880 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-65947b42-09cd-4319-b2ce-c03c90cddacf tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1723 microversion: 2.20 time: 0.167321 Oct 30 01:00:14.104413 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 240/495] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:00:13 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1723 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:00:14.300591 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-16d5db7c-4670-4f7b-896a-0cf3668a6650 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:14.301734 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-16d5db7c-4670-4f7b-896a-0cf3668a6650 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: c7be44ac-3b42-4ace-ac56-65c117e839c6] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:14.309351 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-16d5db7c-4670-4f7b-896a-0cf3668a6650 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:14.309873 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-16d5db7c-4670-4f7b-896a-0cf3668a6650 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:14.310306 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-16d5db7c-4670-4f7b-896a-0cf3668a6650 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:14.311011 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-16d5db7c-4670-4f7b-896a-0cf3668a6650 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:14.311443 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-16d5db7c-4670-4f7b-896a-0cf3668a6650 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:14.312015 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-16d5db7c-4670-4f7b-896a-0cf3668a6650 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:14.443989 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-16d5db7c-4670-4f7b-896a-0cf3668a6650 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:14.452734 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-16d5db7c-4670-4f7b-896a-0cf3668a6650 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6" status: 200 len: 1522 microversion: 2.1 time: 0.159356 Oct 30 01:00:14.453299 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 256/496] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:14 2022] GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6 => generated 1522 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:15.131924 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-2823b149-31be-49f7-9ed3-4894599ddcfa tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:15.132735 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-2823b149-31be-49f7-9ed3-4894599ddcfa tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:15.142626 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2823b149-31be-49f7-9ed3-4894599ddcfa tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:15.143330 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2823b149-31be-49f7-9ed3-4894599ddcfa tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:15.144028 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2823b149-31be-49f7-9ed3-4894599ddcfa tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:15.145044 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2823b149-31be-49f7-9ed3-4894599ddcfa tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:15.145677 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2823b149-31be-49f7-9ed3-4894599ddcfa tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:15.146430 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2823b149-31be-49f7-9ed3-4894599ddcfa tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:15.280709 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-2823b149-31be-49f7-9ed3-4894599ddcfa tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:15.291069 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-2823b149-31be-49f7-9ed3-4894599ddcfa tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:15.292546 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-2823b149-31be-49f7-9ed3-4894599ddcfa tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:15.293708 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-2823b149-31be-49f7-9ed3-4894599ddcfa tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1723 microversion: 2.20 time: 0.167925 Oct 30 01:00:15.294245 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 241/497] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:00:15 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1723 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:00:15.487529 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-898f4aec-b5bc-4fe3-981a-c21b97a4bbbb tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:15.488610 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-898f4aec-b5bc-4fe3-981a-c21b97a4bbbb tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: c7be44ac-3b42-4ace-ac56-65c117e839c6] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:15.498946 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-898f4aec-b5bc-4fe3-981a-c21b97a4bbbb tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:15.499796 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-898f4aec-b5bc-4fe3-981a-c21b97a4bbbb tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:15.500537 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-898f4aec-b5bc-4fe3-981a-c21b97a4bbbb tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:15.501683 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-898f4aec-b5bc-4fe3-981a-c21b97a4bbbb tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:15.502493 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-898f4aec-b5bc-4fe3-981a-c21b97a4bbbb tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:15.503199 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-898f4aec-b5bc-4fe3-981a-c21b97a4bbbb tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:15.643665 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-898f4aec-b5bc-4fe3-981a-c21b97a4bbbb tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:15.651562 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-898f4aec-b5bc-4fe3-981a-c21b97a4bbbb tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6" status: 200 len: 1522 microversion: 2.1 time: 0.170671 Oct 30 01:00:15.652147 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 257/498] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:15 2022] GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6 => generated 1522 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:16.323812 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-f53f77a9-5fd5-4cc3-a613-c2f84bed2e03 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:16.324607 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-f53f77a9-5fd5-4cc3-a613-c2f84bed2e03 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:16.335688 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f53f77a9-5fd5-4cc3-a613-c2f84bed2e03 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:16.336081 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f53f77a9-5fd5-4cc3-a613-c2f84bed2e03 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:16.336510 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f53f77a9-5fd5-4cc3-a613-c2f84bed2e03 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:16.337195 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f53f77a9-5fd5-4cc3-a613-c2f84bed2e03 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:16.337641 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f53f77a9-5fd5-4cc3-a613-c2f84bed2e03 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:16.338150 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f53f77a9-5fd5-4cc3-a613-c2f84bed2e03 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:16.470552 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-f53f77a9-5fd5-4cc3-a613-c2f84bed2e03 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:16.483026 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-f53f77a9-5fd5-4cc3-a613-c2f84bed2e03 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:16.485198 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-f53f77a9-5fd5-4cc3-a613-c2f84bed2e03 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:16.486932 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-f53f77a9-5fd5-4cc3-a613-c2f84bed2e03 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1723 microversion: 2.20 time: 0.169409 Oct 30 01:00:16.487709 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 242/499] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:00:16 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1723 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:00:16.675926 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-f8366c2a-5853-4696-a931-0bec933761a9 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:16.676479 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-f8366c2a-5853-4696-a931-0bec933761a9 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: c7be44ac-3b42-4ace-ac56-65c117e839c6] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:16.683267 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f8366c2a-5853-4696-a931-0bec933761a9 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:16.683867 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f8366c2a-5853-4696-a931-0bec933761a9 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:16.684380 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f8366c2a-5853-4696-a931-0bec933761a9 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:16.685023 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f8366c2a-5853-4696-a931-0bec933761a9 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:16.685564 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f8366c2a-5853-4696-a931-0bec933761a9 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:16.685981 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f8366c2a-5853-4696-a931-0bec933761a9 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:16.826750 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-f8366c2a-5853-4696-a931-0bec933761a9 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:16.835508 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-f8366c2a-5853-4696-a931-0bec933761a9 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6" status: 200 len: 1522 microversion: 2.1 time: 0.163952 Oct 30 01:00:16.847799 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 258/500] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:16 2022] GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6 => generated 1522 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:17.517139 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-c9c15e26-679b-40b6-9733-4098ec94d0a3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:17.518147 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-c9c15e26-679b-40b6-9733-4098ec94d0a3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:17.529015 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c9c15e26-679b-40b6-9733-4098ec94d0a3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:17.529810 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c9c15e26-679b-40b6-9733-4098ec94d0a3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:17.530526 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c9c15e26-679b-40b6-9733-4098ec94d0a3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:17.531656 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c9c15e26-679b-40b6-9733-4098ec94d0a3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:17.532485 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c9c15e26-679b-40b6-9733-4098ec94d0a3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:17.533181 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c9c15e26-679b-40b6-9733-4098ec94d0a3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:17.662447 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-1945b03c-2748-4116-bf7d-4c78710643b3 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:17.662966 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-1945b03c-2748-4116-bf7d-4c78710643b3 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:17.669327 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1945b03c-2748-4116-bf7d-4c78710643b3 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:17.669477 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-c9c15e26-679b-40b6-9733-4098ec94d0a3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:17.669809 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1945b03c-2748-4116-bf7d-4c78710643b3 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:17.670244 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1945b03c-2748-4116-bf7d-4c78710643b3 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:17.670936 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1945b03c-2748-4116-bf7d-4c78710643b3 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:17.671387 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1945b03c-2748-4116-bf7d-4c78710643b3 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:17.671791 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1945b03c-2748-4116-bf7d-4c78710643b3 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:17.677640 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-c9c15e26-679b-40b6-9733-4098ec94d0a3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:17.679064 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-c9c15e26-679b-40b6-9733-4098ec94d0a3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:17.680248 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-c9c15e26-679b-40b6-9733-4098ec94d0a3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1723 microversion: 2.20 time: 0.169712 Oct 30 01:00:17.680806 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 243/501] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:00:17 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1723 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:00:17.694888 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-1945b03c-2748-4116-bf7d-4c78710643b3 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14/ips" status: 200 len: 145 microversion: 2.1 time: 0.036533 Oct 30 01:00:17.695436 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 259/502] 10.222.0.16 () {58 vars in 1248 bytes} [Sun Oct 30 01:00:17 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14/ips => generated 145 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:00:17.863696 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-51ba29a0-09a3-47ed-ae3f-e86234a7280c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:17.864723 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-51ba29a0-09a3-47ed-ae3f-e86234a7280c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: c7be44ac-3b42-4ace-ac56-65c117e839c6] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:17.872464 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-51ba29a0-09a3-47ed-ae3f-e86234a7280c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:17.872878 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-51ba29a0-09a3-47ed-ae3f-e86234a7280c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:17.873302 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-51ba29a0-09a3-47ed-ae3f-e86234a7280c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:17.873983 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-51ba29a0-09a3-47ed-ae3f-e86234a7280c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:17.874425 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-51ba29a0-09a3-47ed-ae3f-e86234a7280c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:17.874831 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-51ba29a0-09a3-47ed-ae3f-e86234a7280c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:18.020818 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-51ba29a0-09a3-47ed-ae3f-e86234a7280c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:18.029309 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-51ba29a0-09a3-47ed-ae3f-e86234a7280c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6" status: 200 len: 1522 microversion: 2.1 time: 0.172463 Oct 30 01:00:18.029882 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 244/503] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:17 2022] GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6 => generated 1522 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:18.706382 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-3b5a35de-287e-4664-a8c7-5764b87e36a6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:18.706987 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-3b5a35de-287e-4664-a8c7-5764b87e36a6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:18.713577 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3b5a35de-287e-4664-a8c7-5764b87e36a6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:18.714058 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3b5a35de-287e-4664-a8c7-5764b87e36a6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:18.714490 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3b5a35de-287e-4664-a8c7-5764b87e36a6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:18.715182 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3b5a35de-287e-4664-a8c7-5764b87e36a6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:18.715631 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3b5a35de-287e-4664-a8c7-5764b87e36a6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:18.716085 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3b5a35de-287e-4664-a8c7-5764b87e36a6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:18.720111 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell1:UPDATE=3,nova_cell1:SELECT=95,nova_cell1:INSERT=2 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:00:18.852478 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-3b5a35de-287e-4664-a8c7-5764b87e36a6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:18.861125 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-3b5a35de-287e-4664-a8c7-5764b87e36a6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:18.862556 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-3b5a35de-287e-4664-a8c7-5764b87e36a6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:18.863801 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-3b5a35de-287e-4664-a8c7-5764b87e36a6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1723 microversion: 2.20 time: 0.161457 Oct 30 01:00:18.864330 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 260/504] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:00:18 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1723 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:00:19.052070 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-4e781ce3-f522-422f-8350-4a31178ce22f tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:19.052861 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-4e781ce3-f522-422f-8350-4a31178ce22f tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: c7be44ac-3b42-4ace-ac56-65c117e839c6] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:19.059614 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4e781ce3-f522-422f-8350-4a31178ce22f tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:19.060151 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4e781ce3-f522-422f-8350-4a31178ce22f tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:19.060565 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4e781ce3-f522-422f-8350-4a31178ce22f tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:19.061260 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4e781ce3-f522-422f-8350-4a31178ce22f tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:19.061702 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4e781ce3-f522-422f-8350-4a31178ce22f tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:19.062142 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4e781ce3-f522-422f-8350-4a31178ce22f tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:19.201175 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-4e781ce3-f522-422f-8350-4a31178ce22f tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:19.211063 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-4e781ce3-f522-422f-8350-4a31178ce22f tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6" status: 200 len: 1522 microversion: 2.1 time: 0.162975 Oct 30 01:00:19.211781 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 245/505] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:19 2022] GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6 => generated 1522 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:19.893884 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-a1a47b75-5f85-4a65-ba89-db863cb1a70e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:19.895001 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-a1a47b75-5f85-4a65-ba89-db863cb1a70e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:19.905374 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a1a47b75-5f85-4a65-ba89-db863cb1a70e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:19.906225 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a1a47b75-5f85-4a65-ba89-db863cb1a70e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:19.906939 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a1a47b75-5f85-4a65-ba89-db863cb1a70e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:19.908145 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a1a47b75-5f85-4a65-ba89-db863cb1a70e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:19.908892 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a1a47b75-5f85-4a65-ba89-db863cb1a70e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:19.909585 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a1a47b75-5f85-4a65-ba89-db863cb1a70e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:20.051022 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-a1a47b75-5f85-4a65-ba89-db863cb1a70e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:20.060681 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-a1a47b75-5f85-4a65-ba89-db863cb1a70e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:20.062106 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-a1a47b75-5f85-4a65-ba89-db863cb1a70e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:20.063278 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-a1a47b75-5f85-4a65-ba89-db863cb1a70e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1723 microversion: 2.20 time: 0.176018 Oct 30 01:00:20.064117 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 261/506] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:00:19 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1723 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:00:20.235495 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-f7979e7a-41f1-492a-b9b8-3781c9695b18 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:20.236144 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-f7979e7a-41f1-492a-b9b8-3781c9695b18 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: c7be44ac-3b42-4ace-ac56-65c117e839c6] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:20.242835 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f7979e7a-41f1-492a-b9b8-3781c9695b18 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:20.243315 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f7979e7a-41f1-492a-b9b8-3781c9695b18 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:20.243995 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f7979e7a-41f1-492a-b9b8-3781c9695b18 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:20.244602 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f7979e7a-41f1-492a-b9b8-3781c9695b18 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:20.245004 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f7979e7a-41f1-492a-b9b8-3781c9695b18 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:20.245488 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f7979e7a-41f1-492a-b9b8-3781c9695b18 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:20.383901 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-f7979e7a-41f1-492a-b9b8-3781c9695b18 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:20.392177 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-f7979e7a-41f1-492a-b9b8-3781c9695b18 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6" status: 200 len: 1522 microversion: 2.1 time: 0.160586 Oct 30 01:00:20.392747 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 246/507] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:20 2022] GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6 => generated 1522 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:21.093665 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-e1c9c617-d567-4293-955f-ab065a5296b6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:21.094706 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-e1c9c617-d567-4293-955f-ab065a5296b6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:21.104701 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e1c9c617-d567-4293-955f-ab065a5296b6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:21.105184 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e1c9c617-d567-4293-955f-ab065a5296b6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:21.105612 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e1c9c617-d567-4293-955f-ab065a5296b6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:21.106279 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e1c9c617-d567-4293-955f-ab065a5296b6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:21.106705 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e1c9c617-d567-4293-955f-ab065a5296b6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:21.107236 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e1c9c617-d567-4293-955f-ab065a5296b6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:21.241007 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-e1c9c617-d567-4293-955f-ab065a5296b6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:21.249599 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-e1c9c617-d567-4293-955f-ab065a5296b6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:21.251051 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-e1c9c617-d567-4293-955f-ab065a5296b6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:21.252264 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-e1c9c617-d567-4293-955f-ab065a5296b6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1723 microversion: 2.20 time: 0.165092 Oct 30 01:00:21.252797 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 262/508] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:00:21 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1723 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:00:21.415049 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-9a76cdf1-f4b3-4c88-9cef-71a047608569 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:21.415772 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-9a76cdf1-f4b3-4c88-9cef-71a047608569 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: c7be44ac-3b42-4ace-ac56-65c117e839c6] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:21.423259 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9a76cdf1-f4b3-4c88-9cef-71a047608569 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:21.423748 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9a76cdf1-f4b3-4c88-9cef-71a047608569 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:21.424211 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9a76cdf1-f4b3-4c88-9cef-71a047608569 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:21.424886 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9a76cdf1-f4b3-4c88-9cef-71a047608569 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:21.425340 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9a76cdf1-f4b3-4c88-9cef-71a047608569 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:21.425760 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9a76cdf1-f4b3-4c88-9cef-71a047608569 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:21.564844 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-9a76cdf1-f4b3-4c88-9cef-71a047608569 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:21.573293 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-9a76cdf1-f4b3-4c88-9cef-71a047608569 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6" status: 200 len: 1522 microversion: 2.1 time: 0.162118 Oct 30 01:00:21.573891 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 247/509] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:21 2022] GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6 => generated 1522 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:22.278044 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-3692b79b-7eee-4bac-bd71-5c22015b6d01 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:22.278641 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-3692b79b-7eee-4bac-bd71-5c22015b6d01 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:22.285109 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3692b79b-7eee-4bac-bd71-5c22015b6d01 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:22.285597 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3692b79b-7eee-4bac-bd71-5c22015b6d01 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:22.286027 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3692b79b-7eee-4bac-bd71-5c22015b6d01 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:22.286778 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3692b79b-7eee-4bac-bd71-5c22015b6d01 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:22.287288 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3692b79b-7eee-4bac-bd71-5c22015b6d01 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:22.287753 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3692b79b-7eee-4bac-bd71-5c22015b6d01 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:22.426482 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-3692b79b-7eee-4bac-bd71-5c22015b6d01 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:22.435544 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-3692b79b-7eee-4bac-bd71-5c22015b6d01 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:22.437017 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-3692b79b-7eee-4bac-bd71-5c22015b6d01 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:22.438168 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-3692b79b-7eee-4bac-bd71-5c22015b6d01 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1723 microversion: 2.20 time: 0.164412 Oct 30 01:00:22.438908 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 263/510] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:00:22 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1723 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:00:22.597512 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-1e7f7e02-fbc0-48eb-b677-a4dbf7d54b3e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:22.598119 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-1e7f7e02-fbc0-48eb-b677-a4dbf7d54b3e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: c7be44ac-3b42-4ace-ac56-65c117e839c6] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:22.605069 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1e7f7e02-fbc0-48eb-b677-a4dbf7d54b3e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:22.605472 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1e7f7e02-fbc0-48eb-b677-a4dbf7d54b3e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:22.605893 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1e7f7e02-fbc0-48eb-b677-a4dbf7d54b3e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:22.606568 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1e7f7e02-fbc0-48eb-b677-a4dbf7d54b3e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:22.607013 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1e7f7e02-fbc0-48eb-b677-a4dbf7d54b3e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:22.607426 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1e7f7e02-fbc0-48eb-b677-a4dbf7d54b3e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:22.746890 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-1e7f7e02-fbc0-48eb-b677-a4dbf7d54b3e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:22.755007 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-1e7f7e02-fbc0-48eb-b677-a4dbf7d54b3e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6" status: 200 len: 1522 microversion: 2.1 time: 0.161355 Oct 30 01:00:22.755606 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 248/511] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:22 2022] GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6 => generated 1522 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:23.464098 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-4e8c52e5-5c1c-4840-a9ce-6377b4741487 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:23.464716 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-4e8c52e5-5c1c-4840-a9ce-6377b4741487 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:23.466620 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_api:SELECT=98,nova_api:UPDATE=2 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:00:23.471969 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4e8c52e5-5c1c-4840-a9ce-6377b4741487 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:23.472457 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4e8c52e5-5c1c-4840-a9ce-6377b4741487 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:23.472892 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4e8c52e5-5c1c-4840-a9ce-6377b4741487 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:23.473594 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4e8c52e5-5c1c-4840-a9ce-6377b4741487 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:23.474053 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4e8c52e5-5c1c-4840-a9ce-6377b4741487 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:23.474480 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4e8c52e5-5c1c-4840-a9ce-6377b4741487 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:23.617561 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-4e8c52e5-5c1c-4840-a9ce-6377b4741487 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:23.630745 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-4e8c52e5-5c1c-4840-a9ce-6377b4741487 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:23.633348 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-4e8c52e5-5c1c-4840-a9ce-6377b4741487 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:23.635768 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-4e8c52e5-5c1c-4840-a9ce-6377b4741487 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1723 microversion: 2.20 time: 0.175488 Oct 30 01:00:23.637291 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 264/512] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:00:23 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1723 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:00:23.778901 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-89f6fd9d-65e8-4a5c-900c-d67995ca1d48 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:23.779596 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-89f6fd9d-65e8-4a5c-900c-d67995ca1d48 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: c7be44ac-3b42-4ace-ac56-65c117e839c6] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:23.787006 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-89f6fd9d-65e8-4a5c-900c-d67995ca1d48 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:23.787637 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-89f6fd9d-65e8-4a5c-900c-d67995ca1d48 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:23.788196 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-89f6fd9d-65e8-4a5c-900c-d67995ca1d48 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:23.788890 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-89f6fd9d-65e8-4a5c-900c-d67995ca1d48 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:23.789664 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-89f6fd9d-65e8-4a5c-900c-d67995ca1d48 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:23.790098 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-89f6fd9d-65e8-4a5c-900c-d67995ca1d48 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:23.932349 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-89f6fd9d-65e8-4a5c-900c-d67995ca1d48 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:23.954339 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-89f6fd9d-65e8-4a5c-900c-d67995ca1d48 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6" status: 200 len: 1522 microversion: 2.1 time: 0.179544 Oct 30 01:00:23.955821 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 249/513] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:23 2022] GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6 => generated 1522 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:24.009246 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-2ba52975-64ef-49d5-b7ed-f53227334417 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "a9d9aa2e-2337-4976-9752-1910f19a8aec"}} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:00:24.012053 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-2ba52975-64ef-49d5-b7ed-f53227334417 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:24.018728 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2ba52975-64ef-49d5-b7ed-f53227334417 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:24.019216 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2ba52975-64ef-49d5-b7ed-f53227334417 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:24.019654 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2ba52975-64ef-49d5-b7ed-f53227334417 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:24.020390 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2ba52975-64ef-49d5-b7ed-f53227334417 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:24.021008 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2ba52975-64ef-49d5-b7ed-f53227334417 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:24.021636 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2ba52975-64ef-49d5-b7ed-f53227334417 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:24.667093 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-0eb8a3fc-7a8c-4a82-a730-1c750fc90099 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:24.667724 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-0eb8a3fc-7a8c-4a82-a730-1c750fc90099 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:24.680183 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0eb8a3fc-7a8c-4a82-a730-1c750fc90099 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:24.681050 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0eb8a3fc-7a8c-4a82-a730-1c750fc90099 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:24.682147 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0eb8a3fc-7a8c-4a82-a730-1c750fc90099 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:24.683643 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0eb8a3fc-7a8c-4a82-a730-1c750fc90099 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:24.684388 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0eb8a3fc-7a8c-4a82-a730-1c750fc90099 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:24.685046 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0eb8a3fc-7a8c-4a82-a730-1c750fc90099 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:24.873614 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-0eb8a3fc-7a8c-4a82-a730-1c750fc90099 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:24.883094 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-0eb8a3fc-7a8c-4a82-a730-1c750fc90099 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:24.885096 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-0eb8a3fc-7a8c-4a82-a730-1c750fc90099 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:24.886269 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-0eb8a3fc-7a8c-4a82-a730-1c750fc90099 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1723 microversion: 2.20 time: 0.223475 Oct 30 01:00:24.886978 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 250/514] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:00:24 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1723 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:00:24.981547 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-f0e896c8-3a95-420f-adf1-f27420e3d88a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:24.982406 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-f0e896c8-3a95-420f-adf1-f27420e3d88a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: c7be44ac-3b42-4ace-ac56-65c117e839c6] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:24.989522 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f0e896c8-3a95-420f-adf1-f27420e3d88a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:24.990290 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f0e896c8-3a95-420f-adf1-f27420e3d88a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:24.990809 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f0e896c8-3a95-420f-adf1-f27420e3d88a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:24.991743 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f0e896c8-3a95-420f-adf1-f27420e3d88a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:24.992287 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f0e896c8-3a95-420f-adf1-f27420e3d88a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:24.992925 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f0e896c8-3a95-420f-adf1-f27420e3d88a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:25.147968 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-f0e896c8-3a95-420f-adf1-f27420e3d88a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:25.156023 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-f0e896c8-3a95-420f-adf1-f27420e3d88a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6" status: 200 len: 1522 microversion: 2.1 time: 0.178606 Oct 30 01:00:25.157112 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 251/515] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:24 2022] GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6 => generated 1522 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:25.916239 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-71e9c6f2-fe34-4f81-ae21-9d491fcddc5d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:25.916939 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-71e9c6f2-fe34-4f81-ae21-9d491fcddc5d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:25.924251 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-71e9c6f2-fe34-4f81-ae21-9d491fcddc5d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:25.924757 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-71e9c6f2-fe34-4f81-ae21-9d491fcddc5d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:25.925208 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-71e9c6f2-fe34-4f81-ae21-9d491fcddc5d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:25.925964 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-71e9c6f2-fe34-4f81-ae21-9d491fcddc5d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:25.926438 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-71e9c6f2-fe34-4f81-ae21-9d491fcddc5d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:25.926872 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-71e9c6f2-fe34-4f81-ae21-9d491fcddc5d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:26.063768 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-71e9c6f2-fe34-4f81-ae21-9d491fcddc5d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:26.072675 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-71e9c6f2-fe34-4f81-ae21-9d491fcddc5d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:26.074101 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-71e9c6f2-fe34-4f81-ae21-9d491fcddc5d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:26.075257 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-71e9c6f2-fe34-4f81-ae21-9d491fcddc5d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1723 microversion: 2.20 time: 0.165765 Oct 30 01:00:26.075839 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 252/516] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:00:25 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1723 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:00:26.182283 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-4ecbb6e6-5d8b-4156-a9b8-4ac9b93e5e3e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:26.182907 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-4ecbb6e6-5d8b-4156-a9b8-4ac9b93e5e3e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: c7be44ac-3b42-4ace-ac56-65c117e839c6] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:26.189347 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4ecbb6e6-5d8b-4156-a9b8-4ac9b93e5e3e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:26.189835 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4ecbb6e6-5d8b-4156-a9b8-4ac9b93e5e3e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:26.190267 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4ecbb6e6-5d8b-4156-a9b8-4ac9b93e5e3e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:26.190967 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4ecbb6e6-5d8b-4156-a9b8-4ac9b93e5e3e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:26.191417 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4ecbb6e6-5d8b-4156-a9b8-4ac9b93e5e3e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:26.191879 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4ecbb6e6-5d8b-4156-a9b8-4ac9b93e5e3e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:26.322210 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-4ecbb6e6-5d8b-4156-a9b8-4ac9b93e5e3e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:26.325640 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell1:SELECT=100 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:00:26.330577 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-4ecbb6e6-5d8b-4156-a9b8-4ac9b93e5e3e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6" status: 200 len: 1522 microversion: 2.1 time: 0.153933 Oct 30 01:00:26.331017 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 253/517] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:26 2022] GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6 => generated 1522 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:27.107390 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-7e487788-c04c-40ed-82d1-f4bd0b143dd9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:27.109240 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-7e487788-c04c-40ed-82d1-f4bd0b143dd9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:27.118518 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7e487788-c04c-40ed-82d1-f4bd0b143dd9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:27.119484 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7e487788-c04c-40ed-82d1-f4bd0b143dd9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:27.120098 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7e487788-c04c-40ed-82d1-f4bd0b143dd9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:27.121258 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7e487788-c04c-40ed-82d1-f4bd0b143dd9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:27.122031 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7e487788-c04c-40ed-82d1-f4bd0b143dd9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:27.122761 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7e487788-c04c-40ed-82d1-f4bd0b143dd9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:27.298735 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-7e487788-c04c-40ed-82d1-f4bd0b143dd9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:27.302606 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-7e487788-c04c-40ed-82d1-f4bd0b143dd9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:27.304358 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-7e487788-c04c-40ed-82d1-f4bd0b143dd9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:27.305637 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-7e487788-c04c-40ed-82d1-f4bd0b143dd9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1723 microversion: 2.20 time: 0.203660 Oct 30 01:00:27.315768 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 254/518] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:00:27 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1723 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 30 01:00:27.358715 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-bc722336-02f1-408d-8d87-8b5d72982531 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:27.359329 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-bc722336-02f1-408d-8d87-8b5d72982531 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: c7be44ac-3b42-4ace-ac56-65c117e839c6] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:27.366086 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bc722336-02f1-408d-8d87-8b5d72982531 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:27.366575 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bc722336-02f1-408d-8d87-8b5d72982531 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:27.367097 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bc722336-02f1-408d-8d87-8b5d72982531 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:27.370357 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bc722336-02f1-408d-8d87-8b5d72982531 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:27.370823 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bc722336-02f1-408d-8d87-8b5d72982531 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:27.371248 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bc722336-02f1-408d-8d87-8b5d72982531 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:27.527148 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-bc722336-02f1-408d-8d87-8b5d72982531 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:27.535263 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-bc722336-02f1-408d-8d87-8b5d72982531 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6" status: 200 len: 1522 microversion: 2.1 time: 0.181957 Oct 30 01:00:27.535888 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 255/519] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:27 2022] GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6 => generated 1522 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:27.547833 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-18df75fa-c415-43b2-893c-689857180409 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:27.547833 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-18df75fa-c415-43b2-893c-689857180409 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5652be64-c2a3-4b24-a3aa-c962f204c06e] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:27.548496 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-18df75fa-c415-43b2-893c-689857180409 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:27.549014 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-18df75fa-c415-43b2-893c-689857180409 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:27.549452 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-18df75fa-c415-43b2-893c-689857180409 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:27.552605 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-18df75fa-c415-43b2-893c-689857180409 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:27.552605 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-18df75fa-c415-43b2-893c-689857180409 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:27.552605 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-18df75fa-c415-43b2-893c-689857180409 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:27.577727 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-18df75fa-c415-43b2-893c-689857180409 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e/ips" status: 200 len: 149 microversion: 2.1 time: 0.040737 Oct 30 01:00:27.578798 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 256/520] 10.222.0.16 () {58 vars in 1248 bytes} [Sun Oct 30 01:00:27 2022] GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e/ips => generated 149 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:00:28.338468 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-d6e5fcb7-e969-47da-bb2b-b93d419cc690 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:28.339165 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-d6e5fcb7-e969-47da-bb2b-b93d419cc690 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:28.348990 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d6e5fcb7-e969-47da-bb2b-b93d419cc690 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:28.354402 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d6e5fcb7-e969-47da-bb2b-b93d419cc690 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:28.356307 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d6e5fcb7-e969-47da-bb2b-b93d419cc690 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:28.359410 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d6e5fcb7-e969-47da-bb2b-b93d419cc690 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:28.360186 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d6e5fcb7-e969-47da-bb2b-b93d419cc690 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:28.360966 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d6e5fcb7-e969-47da-bb2b-b93d419cc690 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:28.497513 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-2ba52975-64ef-49d5-b7ed-f53227334417 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "POST /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 4.492326 Oct 30 01:00:28.505320 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 265/521] 10.222.0.16 () {60 vars in 1323 bytes} [Sun Oct 30 01:00:24 2022] POST /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14/os-volume_attachments => generated 194 bytes in 4495 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:00:28.563914 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-d6e5fcb7-e969-47da-bb2b-b93d419cc690 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:28.564258 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-4b8ed765-7a5b-47c9-a42d-4fb085dd3ec8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:28.565171 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-4b8ed765-7a5b-47c9-a42d-4fb085dd3ec8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: c7be44ac-3b42-4ace-ac56-65c117e839c6] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:28.574629 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4b8ed765-7a5b-47c9-a42d-4fb085dd3ec8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:28.575289 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4b8ed765-7a5b-47c9-a42d-4fb085dd3ec8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:28.576605 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4b8ed765-7a5b-47c9-a42d-4fb085dd3ec8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:28.576910 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-d6e5fcb7-e969-47da-bb2b-b93d419cc690 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:28.578329 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-d6e5fcb7-e969-47da-bb2b-b93d419cc690 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:28.579161 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4b8ed765-7a5b-47c9-a42d-4fb085dd3ec8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:28.579820 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-d6e5fcb7-e969-47da-bb2b-b93d419cc690 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1723 microversion: 2.20 time: 0.245009 Oct 30 01:00:28.580329 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4b8ed765-7a5b-47c9-a42d-4fb085dd3ec8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:28.580913 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4b8ed765-7a5b-47c9-a42d-4fb085dd3ec8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:28.584355 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 257/522] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:00:28 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1723 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:00:28.781485 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-4b8ed765-7a5b-47c9-a42d-4fb085dd3ec8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:28.789728 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-4b8ed765-7a5b-47c9-a42d-4fb085dd3ec8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6" status: 200 len: 1522 microversion: 2.1 time: 0.232582 Oct 30 01:00:28.795049 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 266/523] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:28 2022] GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6 => generated 1522 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 01:00:29.610505 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-c69c2105-255a-403e-a98d-5b0e85c10262 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:29.611865 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-c69c2105-255a-403e-a98d-5b0e85c10262 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:29.618871 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c69c2105-255a-403e-a98d-5b0e85c10262 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:29.620999 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c69c2105-255a-403e-a98d-5b0e85c10262 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:29.621654 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c69c2105-255a-403e-a98d-5b0e85c10262 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:29.622602 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c69c2105-255a-403e-a98d-5b0e85c10262 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:29.623355 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c69c2105-255a-403e-a98d-5b0e85c10262 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:29.624012 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c69c2105-255a-403e-a98d-5b0e85c10262 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:29.725767 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-79883cee-f5e8-48d0-95e8-ab87b733846c tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-1113007368", "imageRef": "4c16f4f6-84cb-4463-b82c-b7c65f3826f0", "flavorRef": "452", "networks": [{"uuid": "58afba65-fd3f-46c8-9a69-fc8ae076237c"}]}} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:00:29.784265 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.network.neutron [None req-79883cee-f5e8-48d0-95e8-ab87b733846c tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] validate_networks() for [('58afba65-fd3f-46c8-9a69-fc8ae076237c', None, None, None, None, None)] {{(pid=118961) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 30 01:00:29.787937 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-c69c2105-255a-403e-a98d-5b0e85c10262 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:29.796826 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-c69c2105-255a-403e-a98d-5b0e85c10262 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:29.798333 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-c69c2105-255a-403e-a98d-5b0e85c10262 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:29.799590 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-c69c2105-255a-403e-a98d-5b0e85c10262 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1723 microversion: 2.20 time: 0.193470 Oct 30 01:00:29.800429 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 258/524] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:00:29 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1723 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:00:29.820820 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-42daa64b-ba39-4760-81d1-5a99b384a9a4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:29.821771 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-42daa64b-ba39-4760-81d1-5a99b384a9a4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: c7be44ac-3b42-4ace-ac56-65c117e839c6] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:29.833678 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-42daa64b-ba39-4760-81d1-5a99b384a9a4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:29.834466 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-42daa64b-ba39-4760-81d1-5a99b384a9a4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:29.835108 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-42daa64b-ba39-4760-81d1-5a99b384a9a4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:29.836686 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-42daa64b-ba39-4760-81d1-5a99b384a9a4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:29.837359 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-42daa64b-ba39-4760-81d1-5a99b384a9a4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:29.838177 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-42daa64b-ba39-4760-81d1-5a99b384a9a4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:30.117923 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-42daa64b-ba39-4760-81d1-5a99b384a9a4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:30.129746 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-42daa64b-ba39-4760-81d1-5a99b384a9a4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6" status: 200 len: 1522 microversion: 2.1 time: 0.314789 Oct 30 01:00:30.146185 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 259/525] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:29 2022] GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6 => generated 1522 bytes in 331 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:30.156535 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-9afec39c-a40a-4d38-97d5-cc8df3c94b72 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "00c66df2-3c81-4adf-8028-8c47eb8b88ba"}} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:00:30.160486 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-9afec39c-a40a-4d38-97d5-cc8df3c94b72 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5652be64-c2a3-4b24-a3aa-c962f204c06e] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:30.168039 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9afec39c-a40a-4d38-97d5-cc8df3c94b72 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:30.168726 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9afec39c-a40a-4d38-97d5-cc8df3c94b72 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:30.169278 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9afec39c-a40a-4d38-97d5-cc8df3c94b72 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:30.170113 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9afec39c-a40a-4d38-97d5-cc8df3c94b72 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:30.170701 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9afec39c-a40a-4d38-97d5-cc8df3c94b72 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:30.171373 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9afec39c-a40a-4d38-97d5-cc8df3c94b72 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:30.376331 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-79883cee-f5e8-48d0-95e8-ab87b733846c tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Flavor limits 0:0:0 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 30 01:00:30.377209 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-79883cee-f5e8-48d0-95e8-ab87b733846c tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Image limits 0:0:0 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 30 01:00:30.380492 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-79883cee-f5e8-48d0-95e8-ab87b733846c tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Flavor pref 0:0:0 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 30 01:00:30.381103 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-79883cee-f5e8-48d0-95e8-ab87b733846c tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Image pref 0:0:0 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 30 01:00:30.381648 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-79883cee-f5e8-48d0-95e8-ab87b733846c tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 30 01:00:30.392058 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-79883cee-f5e8-48d0-95e8-ab87b733846c tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:30.393085 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-79883cee-f5e8-48d0-95e8-ab87b733846c tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:30.393636 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-79883cee-f5e8-48d0-95e8-ab87b733846c tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:30.394615 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-79883cee-f5e8-48d0-95e8-ab87b733846c tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:30.395280 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-79883cee-f5e8-48d0-95e8-ab87b733846c tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:30.395828 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-79883cee-f5e8-48d0-95e8-ab87b733846c tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:30.414728 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.quota [None req-79883cee-f5e8-48d0-95e8-ab87b733846c tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Getting quotas for project 2fbc73a853584aaba6109090bd6b1281. Resources: {'ram', 'instances', 'cores'} {{(pid=118961) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 30 01:00:30.425909 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.quota [None req-79883cee-f5e8-48d0-95e8-ab87b733846c tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Getting quotas for user ddeaa7a90ee7483590e6bc0d189e1e2a and project 2fbc73a853584aaba6109090bd6b1281. Resources: {'ram', 'instances', 'cores'} {{(pid=118961) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 30 01:00:30.442758 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-79883cee-f5e8-48d0-95e8-ab87b733846c tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Going to run 1 instances... {{(pid=118961) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 30 01:00:30.446340 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-79883cee-f5e8-48d0-95e8-ab87b733846c tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: db9ac414-31a5-4570-aa3f-37db3dac2049] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4c16f4f6-84cb-4463-b82c-b7c65f3826f0',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118961) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 30 01:00:30.527540 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-79883cee-f5e8-48d0-95e8-ab87b733846c tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.805460 Oct 30 01:00:30.528373 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 267/526] 10.222.0.16 () {60 vars in 1147 bytes} [Sun Oct 30 01:00:29 2022] POST /compute/v2.1/servers => generated 380 bytes in 807 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 30 01:00:30.551351 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-9eb2adb5-fd2a-40ab-a747-5eada88204da tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:30.552469 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-9eb2adb5-fd2a-40ab-a747-5eada88204da tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: db9ac414-31a5-4570-aa3f-37db3dac2049] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:30.579809 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-9afec39c-a40a-4d38-97d5-cc8df3c94b72 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "POST /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.427808 Oct 30 01:00:30.585828 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 260/527] 10.222.0.16 () {60 vars in 1323 bytes} [Sun Oct 30 01:00:30 2022] POST /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e/os-volume_attachments => generated 194 bytes in 432 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:00:30.643254 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-9eb2adb5-fd2a-40ab-a747-5eada88204da tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:30.651393 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-9eb2adb5-fd2a-40ab-a747-5eada88204da tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049" status: 200 len: 1219 microversion: 2.1 time: 0.115530 Oct 30 01:00:30.660847 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 268/528] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:30 2022] GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049 => generated 1219 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 01:00:30.827213 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-6bcd6b4e-bef5-4650-8e52-43639e9c75ce tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:30.829189 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-6bcd6b4e-bef5-4650-8e52-43639e9c75ce tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:30.848827 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6bcd6b4e-bef5-4650-8e52-43639e9c75ce tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:30.850596 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6bcd6b4e-bef5-4650-8e52-43639e9c75ce tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:30.851382 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6bcd6b4e-bef5-4650-8e52-43639e9c75ce tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:30.854319 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6bcd6b4e-bef5-4650-8e52-43639e9c75ce tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:30.854964 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6bcd6b4e-bef5-4650-8e52-43639e9c75ce tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:30.856970 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6bcd6b4e-bef5-4650-8e52-43639e9c75ce tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:31.056707 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-6bcd6b4e-bef5-4650-8e52-43639e9c75ce tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:31.069834 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-6bcd6b4e-bef5-4650-8e52-43639e9c75ce tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:31.072335 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-6bcd6b4e-bef5-4650-8e52-43639e9c75ce tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:31.073923 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-6bcd6b4e-bef5-4650-8e52-43639e9c75ce tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1723 microversion: 2.20 time: 0.252238 Oct 30 01:00:31.074713 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 261/529] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:00:30 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1723 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:00:31.160152 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-04b86e23-1638-42a3-bdd0-a364837bcc10 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:31.161615 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-04b86e23-1638-42a3-bdd0-a364837bcc10 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: c7be44ac-3b42-4ace-ac56-65c117e839c6] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:31.168844 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-04b86e23-1638-42a3-bdd0-a364837bcc10 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:31.170315 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-04b86e23-1638-42a3-bdd0-a364837bcc10 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:31.171004 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-04b86e23-1638-42a3-bdd0-a364837bcc10 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:31.171990 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-04b86e23-1638-42a3-bdd0-a364837bcc10 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:31.172673 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-04b86e23-1638-42a3-bdd0-a364837bcc10 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:31.173694 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-04b86e23-1638-42a3-bdd0-a364837bcc10 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:31.356377 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-04b86e23-1638-42a3-bdd0-a364837bcc10 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:31.365831 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-04b86e23-1638-42a3-bdd0-a364837bcc10 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6" status: 200 len: 1522 microversion: 2.1 time: 0.210540 Oct 30 01:00:31.391900 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 269/530] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:31 2022] GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6 => generated 1522 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:31.680121 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-e6c5b1e7-1596-4f65-83a2-1d084a75b99e tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:31.681268 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-e6c5b1e7-1596-4f65-83a2-1d084a75b99e tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: db9ac414-31a5-4570-aa3f-37db3dac2049] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:31.690725 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e6c5b1e7-1596-4f65-83a2-1d084a75b99e tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:31.691900 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e6c5b1e7-1596-4f65-83a2-1d084a75b99e tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:31.692520 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e6c5b1e7-1596-4f65-83a2-1d084a75b99e tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:31.693663 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e6c5b1e7-1596-4f65-83a2-1d084a75b99e tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:31.696079 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e6c5b1e7-1596-4f65-83a2-1d084a75b99e tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:31.696079 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e6c5b1e7-1596-4f65-83a2-1d084a75b99e tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:31.808857 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-e6c5b1e7-1596-4f65-83a2-1d084a75b99e tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:31.817502 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-e6c5b1e7-1596-4f65-83a2-1d084a75b99e tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049" status: 200 len: 1271 microversion: 2.1 time: 0.141471 Oct 30 01:00:31.818399 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 262/531] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:31 2022] GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049 => generated 1271 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:32.100728 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-39e811b9-3119-4fcf-83d6-83868d325fff tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:32.101817 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-39e811b9-3119-4fcf-83d6-83868d325fff tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:32.109310 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-39e811b9-3119-4fcf-83d6-83868d325fff tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:32.109898 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-39e811b9-3119-4fcf-83d6-83868d325fff tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:32.110621 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-39e811b9-3119-4fcf-83d6-83868d325fff tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:32.111315 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-39e811b9-3119-4fcf-83d6-83868d325fff tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:32.111787 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-39e811b9-3119-4fcf-83d6-83868d325fff tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:32.112429 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-39e811b9-3119-4fcf-83d6-83868d325fff tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:32.252751 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-39e811b9-3119-4fcf-83d6-83868d325fff tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:32.261723 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-39e811b9-3119-4fcf-83d6-83868d325fff tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:32.263207 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-39e811b9-3119-4fcf-83d6-83868d325fff tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:32.264523 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-39e811b9-3119-4fcf-83d6-83868d325fff tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1723 microversion: 2.20 time: 0.168000 Oct 30 01:00:32.265370 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 270/532] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:00:32 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1723 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:00:32.388949 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-50d0bdf8-b930-42bb-b9ae-5f0ca79c6b49 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:32.390307 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-50d0bdf8-b930-42bb-b9ae-5f0ca79c6b49 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: c7be44ac-3b42-4ace-ac56-65c117e839c6] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:32.397422 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-50d0bdf8-b930-42bb-b9ae-5f0ca79c6b49 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:32.398330 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-50d0bdf8-b930-42bb-b9ae-5f0ca79c6b49 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:32.398964 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-50d0bdf8-b930-42bb-b9ae-5f0ca79c6b49 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:32.399930 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-50d0bdf8-b930-42bb-b9ae-5f0ca79c6b49 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:32.400687 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-50d0bdf8-b930-42bb-b9ae-5f0ca79c6b49 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:32.401402 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-50d0bdf8-b930-42bb-b9ae-5f0ca79c6b49 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:32.580789 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-50d0bdf8-b930-42bb-b9ae-5f0ca79c6b49 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:32.589262 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-50d0bdf8-b930-42bb-b9ae-5f0ca79c6b49 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6" status: 200 len: 1522 microversion: 2.1 time: 0.204521 Oct 30 01:00:32.604257 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 263/533] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:32 2022] GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6 => generated 1522 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:32.845759 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-4a3dd932-d3a2-4ede-a6f4-f9cce63889bf tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:32.846466 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-4a3dd932-d3a2-4ede-a6f4-f9cce63889bf tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: db9ac414-31a5-4570-aa3f-37db3dac2049] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:32.853790 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4a3dd932-d3a2-4ede-a6f4-f9cce63889bf tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:32.854331 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4a3dd932-d3a2-4ede-a6f4-f9cce63889bf tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:32.854851 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4a3dd932-d3a2-4ede-a6f4-f9cce63889bf tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:32.855560 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4a3dd932-d3a2-4ede-a6f4-f9cce63889bf tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:32.856040 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4a3dd932-d3a2-4ede-a6f4-f9cce63889bf tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:32.856458 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4a3dd932-d3a2-4ede-a6f4-f9cce63889bf tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:32.939021 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-4a3dd932-d3a2-4ede-a6f4-f9cce63889bf tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:32.951201 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-4a3dd932-d3a2-4ede-a6f4-f9cce63889bf tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049" status: 200 len: 1271 microversion: 2.1 time: 0.108891 Oct 30 01:00:32.952578 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 271/534] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:32 2022] GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049 => generated 1271 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:33.291334 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-346366b7-3e51-4451-b2b2-b784ec379ada tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:33.292669 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-346366b7-3e51-4451-b2b2-b784ec379ada tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:33.299271 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-346366b7-3e51-4451-b2b2-b784ec379ada tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:33.299780 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-346366b7-3e51-4451-b2b2-b784ec379ada tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:33.300396 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-346366b7-3e51-4451-b2b2-b784ec379ada tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:33.301306 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-346366b7-3e51-4451-b2b2-b784ec379ada tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:33.301957 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-346366b7-3e51-4451-b2b2-b784ec379ada tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:33.302552 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-346366b7-3e51-4451-b2b2-b784ec379ada tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:33.445815 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-346366b7-3e51-4451-b2b2-b784ec379ada tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:33.458530 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-346366b7-3e51-4451-b2b2-b784ec379ada tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:33.459992 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-346366b7-3e51-4451-b2b2-b784ec379ada tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:33.461372 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-346366b7-3e51-4451-b2b2-b784ec379ada tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1723 microversion: 2.20 time: 0.174176 Oct 30 01:00:33.462084 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 264/535] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:00:33 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1723 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:00:33.614399 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-8650c4e7-6e70-4749-b8db-f5a63619351a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:33.615135 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-8650c4e7-6e70-4749-b8db-f5a63619351a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: c7be44ac-3b42-4ace-ac56-65c117e839c6] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:33.624759 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8650c4e7-6e70-4749-b8db-f5a63619351a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:33.625256 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8650c4e7-6e70-4749-b8db-f5a63619351a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:33.625695 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8650c4e7-6e70-4749-b8db-f5a63619351a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:33.626401 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8650c4e7-6e70-4749-b8db-f5a63619351a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:33.627234 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8650c4e7-6e70-4749-b8db-f5a63619351a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:33.627589 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8650c4e7-6e70-4749-b8db-f5a63619351a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:33.774062 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-8650c4e7-6e70-4749-b8db-f5a63619351a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:33.782630 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-8650c4e7-6e70-4749-b8db-f5a63619351a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6" status: 200 len: 1522 microversion: 2.1 time: 0.172123 Oct 30 01:00:33.783553 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 272/536] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:33 2022] GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6 => generated 1522 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:33.974563 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-90368fca-0a6f-4ef2-9778-812667ecd81e tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:33.975170 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-90368fca-0a6f-4ef2-9778-812667ecd81e tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: db9ac414-31a5-4570-aa3f-37db3dac2049] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:33.982357 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-90368fca-0a6f-4ef2-9778-812667ecd81e tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:33.983096 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-90368fca-0a6f-4ef2-9778-812667ecd81e tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:33.983626 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-90368fca-0a6f-4ef2-9778-812667ecd81e tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:33.984425 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-90368fca-0a6f-4ef2-9778-812667ecd81e tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:33.985088 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-90368fca-0a6f-4ef2-9778-812667ecd81e tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:33.986019 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-90368fca-0a6f-4ef2-9778-812667ecd81e tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:34.067054 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-90368fca-0a6f-4ef2-9778-812667ecd81e tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:34.075383 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-90368fca-0a6f-4ef2-9778-812667ecd81e tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049" status: 200 len: 1271 microversion: 2.1 time: 0.104694 Oct 30 01:00:34.076686 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 265/537] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:33 2022] GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049 => generated 1271 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:34.489196 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-a6bd3f00-e35a-482e-beaa-eb4cb3390f53 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:34.491187 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-a6bd3f00-e35a-482e-beaa-eb4cb3390f53 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:34.502064 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a6bd3f00-e35a-482e-beaa-eb4cb3390f53 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:34.503078 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a6bd3f00-e35a-482e-beaa-eb4cb3390f53 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:34.503746 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a6bd3f00-e35a-482e-beaa-eb4cb3390f53 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:34.504671 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a6bd3f00-e35a-482e-beaa-eb4cb3390f53 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:34.505312 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a6bd3f00-e35a-482e-beaa-eb4cb3390f53 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:34.505929 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a6bd3f00-e35a-482e-beaa-eb4cb3390f53 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:34.712393 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-a6bd3f00-e35a-482e-beaa-eb4cb3390f53 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:34.721778 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-a6bd3f00-e35a-482e-beaa-eb4cb3390f53 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:34.723220 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-a6bd3f00-e35a-482e-beaa-eb4cb3390f53 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:34.724462 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-a6bd3f00-e35a-482e-beaa-eb4cb3390f53 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1723 microversion: 2.20 time: 0.240492 Oct 30 01:00:34.740034 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 273/538] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:00:34 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1723 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:00:34.806599 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-d572d98e-d6f2-43c0-b28a-e96f2309c620 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:34.808479 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-d572d98e-d6f2-43c0-b28a-e96f2309c620 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: c7be44ac-3b42-4ace-ac56-65c117e839c6] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:34.820523 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d572d98e-d6f2-43c0-b28a-e96f2309c620 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:34.821027 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d572d98e-d6f2-43c0-b28a-e96f2309c620 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:34.821836 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d572d98e-d6f2-43c0-b28a-e96f2309c620 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:34.823277 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d572d98e-d6f2-43c0-b28a-e96f2309c620 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:34.823928 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d572d98e-d6f2-43c0-b28a-e96f2309c620 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:34.824572 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d572d98e-d6f2-43c0-b28a-e96f2309c620 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:35.008901 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-d572d98e-d6f2-43c0-b28a-e96f2309c620 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:35.020635 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-d572d98e-d6f2-43c0-b28a-e96f2309c620 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6" status: 200 len: 1522 microversion: 2.1 time: 0.218237 Oct 30 01:00:35.027774 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 266/539] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:34 2022] GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6 => generated 1522 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:35.102766 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-cfa72df2-2408-46b4-896d-a76faf4478c6 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:35.104121 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-cfa72df2-2408-46b4-896d-a76faf4478c6 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: db9ac414-31a5-4570-aa3f-37db3dac2049] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:35.113626 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cfa72df2-2408-46b4-896d-a76faf4478c6 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:35.116700 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cfa72df2-2408-46b4-896d-a76faf4478c6 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:35.117615 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cfa72df2-2408-46b4-896d-a76faf4478c6 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:35.124002 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cfa72df2-2408-46b4-896d-a76faf4478c6 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:35.124760 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cfa72df2-2408-46b4-896d-a76faf4478c6 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:35.125393 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cfa72df2-2408-46b4-896d-a76faf4478c6 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:35.223866 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-cfa72df2-2408-46b4-896d-a76faf4478c6 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:35.240812 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-cfa72df2-2408-46b4-896d-a76faf4478c6 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049" status: 200 len: 1277 microversion: 2.1 time: 0.143078 Oct 30 01:00:35.242163 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 274/540] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:35 2022] GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049 => generated 1277 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:35.750279 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-6a86fb31-62c2-4acb-ad14-7e550d171dfe tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:35.751267 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-6a86fb31-62c2-4acb-ad14-7e550d171dfe tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:35.757851 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6a86fb31-62c2-4acb-ad14-7e550d171dfe tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:35.758651 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6a86fb31-62c2-4acb-ad14-7e550d171dfe tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:35.759451 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6a86fb31-62c2-4acb-ad14-7e550d171dfe tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:35.760326 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6a86fb31-62c2-4acb-ad14-7e550d171dfe tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:35.761098 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6a86fb31-62c2-4acb-ad14-7e550d171dfe tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:35.761585 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6a86fb31-62c2-4acb-ad14-7e550d171dfe tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:35.949850 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-6a86fb31-62c2-4acb-ad14-7e550d171dfe tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:35.961600 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-6a86fb31-62c2-4acb-ad14-7e550d171dfe tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:35.963957 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-6a86fb31-62c2-4acb-ad14-7e550d171dfe tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:35.965652 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-6a86fb31-62c2-4acb-ad14-7e550d171dfe tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1723 microversion: 2.20 time: 0.219374 Oct 30 01:00:35.966374 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 267/541] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:00:35 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1723 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:00:36.047978 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-ec427594-3e2d-48a9-af69-04032db17146 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:36.048695 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-ec427594-3e2d-48a9-af69-04032db17146 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: c7be44ac-3b42-4ace-ac56-65c117e839c6] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:36.064368 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ec427594-3e2d-48a9-af69-04032db17146 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:36.064955 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ec427594-3e2d-48a9-af69-04032db17146 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:36.066173 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ec427594-3e2d-48a9-af69-04032db17146 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:36.066975 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ec427594-3e2d-48a9-af69-04032db17146 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:36.069310 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ec427594-3e2d-48a9-af69-04032db17146 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:36.069889 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ec427594-3e2d-48a9-af69-04032db17146 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:36.273634 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-62e8fa05-ef1c-4b5c-9066-4d7e62c0b74b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:36.274481 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-62e8fa05-ef1c-4b5c-9066-4d7e62c0b74b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: db9ac414-31a5-4570-aa3f-37db3dac2049] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:36.284758 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-62e8fa05-ef1c-4b5c-9066-4d7e62c0b74b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:36.286371 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-ec427594-3e2d-48a9-af69-04032db17146 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:36.287589 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-62e8fa05-ef1c-4b5c-9066-4d7e62c0b74b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:36.291747 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-62e8fa05-ef1c-4b5c-9066-4d7e62c0b74b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:36.292595 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-62e8fa05-ef1c-4b5c-9066-4d7e62c0b74b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:36.293323 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-62e8fa05-ef1c-4b5c-9066-4d7e62c0b74b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:36.293831 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-62e8fa05-ef1c-4b5c-9066-4d7e62c0b74b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:36.298112 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-ec427594-3e2d-48a9-af69-04032db17146 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6" status: 200 len: 1522 microversion: 2.1 time: 0.254837 Oct 30 01:00:36.321859 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 275/542] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:36 2022] GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6 => generated 1522 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:36.449793 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-62e8fa05-ef1c-4b5c-9066-4d7e62c0b74b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:36.458704 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-62e8fa05-ef1c-4b5c-9066-4d7e62c0b74b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049" status: 200 len: 1319 microversion: 2.1 time: 0.190147 Oct 30 01:00:36.459967 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 268/543] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:36 2022] GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049 => generated 1319 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 01:00:36.994035 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-07ec7e85-f009-447f-9f80-b732fcd02b7f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:36.995690 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-07ec7e85-f009-447f-9f80-b732fcd02b7f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:37.003406 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-07ec7e85-f009-447f-9f80-b732fcd02b7f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:37.004295 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-07ec7e85-f009-447f-9f80-b732fcd02b7f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:37.005023 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-07ec7e85-f009-447f-9f80-b732fcd02b7f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:37.005980 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-07ec7e85-f009-447f-9f80-b732fcd02b7f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:37.006624 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-07ec7e85-f009-447f-9f80-b732fcd02b7f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:37.007245 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-07ec7e85-f009-447f-9f80-b732fcd02b7f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:37.242191 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-07ec7e85-f009-447f-9f80-b732fcd02b7f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:37.256586 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-07ec7e85-f009-447f-9f80-b732fcd02b7f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:37.259440 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-07ec7e85-f009-447f-9f80-b732fcd02b7f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:37.261724 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-07ec7e85-f009-447f-9f80-b732fcd02b7f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1744 microversion: 2.20 time: 0.274385 Oct 30 01:00:37.262383 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 276/544] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:00:36 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1744 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:00:37.328921 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-06d91188-c50c-407a-b370-5e5234118593 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:37.329539 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-06d91188-c50c-407a-b370-5e5234118593 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: c7be44ac-3b42-4ace-ac56-65c117e839c6] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:37.347286 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-06d91188-c50c-407a-b370-5e5234118593 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:37.355884 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-06d91188-c50c-407a-b370-5e5234118593 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:37.356366 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-06d91188-c50c-407a-b370-5e5234118593 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:37.360039 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-06d91188-c50c-407a-b370-5e5234118593 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:37.360505 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-06d91188-c50c-407a-b370-5e5234118593 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:37.360952 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-06d91188-c50c-407a-b370-5e5234118593 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:37.418749 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:00:37.492967 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-a5a80f34-8a38-4b32-aa97-f39c15e76610 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:37.494508 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-a5a80f34-8a38-4b32-aa97-f39c15e76610 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: db9ac414-31a5-4570-aa3f-37db3dac2049] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:37.516764 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a5a80f34-8a38-4b32-aa97-f39c15e76610 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:37.521957 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a5a80f34-8a38-4b32-aa97-f39c15e76610 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:37.522749 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a5a80f34-8a38-4b32-aa97-f39c15e76610 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:37.523687 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a5a80f34-8a38-4b32-aa97-f39c15e76610 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:37.530298 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a5a80f34-8a38-4b32-aa97-f39c15e76610 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:37.531065 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a5a80f34-8a38-4b32-aa97-f39c15e76610 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:37.536099 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:00:37.958788 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-06d91188-c50c-407a-b370-5e5234118593 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:37.968730 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-06d91188-c50c-407a-b370-5e5234118593 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6" status: 200 len: 1522 microversion: 2.1 time: 0.643811 Oct 30 01:00:37.969551 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 269/545] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:37 2022] GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6 => generated 1522 bytes in 645 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:37.992455 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [req-a563c9bd-f9d5-4e34-b0c0-73f3cefc9f28 req-b38cd824-6890-41b9-aba9-1589213dea7a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "db9ac414-31a5-4570-aa3f-37db3dac2049", "tag": "56471ebd-e8c3-4ce0-bd02-87518e117371"}]} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:00:38.008310 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-a563c9bd-f9d5-4e34-b0c0-73f3cefc9f28 req-b38cd824-6890-41b9-aba9-1589213dea7a service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:38.009072 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-a563c9bd-f9d5-4e34-b0c0-73f3cefc9f28 req-b38cd824-6890-41b9-aba9-1589213dea7a service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:38.009725 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-a563c9bd-f9d5-4e34-b0c0-73f3cefc9f28 req-b38cd824-6890-41b9-aba9-1589213dea7a service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:38.025482 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.compute.server_external_events [req-a563c9bd-f9d5-4e34-b0c0-73f3cefc9f28 req-b38cd824-6890-41b9-aba9-1589213dea7a service nova] Creating event network-changed:56471ebd-e8c3-4ce0-bd02-87518e117371 for instance db9ac414-31a5-4570-aa3f-37db3dac2049 on hv-cinder-maste Oct 30 01:00:38.031845 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [req-a563c9bd-f9d5-4e34-b0c0-73f3cefc9f28 req-b38cd824-6890-41b9-aba9-1589213dea7a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.046730 Oct 30 01:00:38.037756 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 270/546] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:00:37 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:00:38.044980 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-a5a80f34-8a38-4b32-aa97-f39c15e76610 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:38.053435 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-a5a80f34-8a38-4b32-aa97-f39c15e76610 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049" status: 200 len: 1319 microversion: 2.1 time: 0.570673 Oct 30 01:00:38.054687 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 277/547] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:37 2022] GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049 => generated 1319 bytes in 573 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:38.289964 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-592811bf-8692-45dd-9586-7836d5b6de15 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:38.293908 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-592811bf-8692-45dd-9586-7836d5b6de15 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:38.308479 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-592811bf-8692-45dd-9586-7836d5b6de15 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:38.309787 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-592811bf-8692-45dd-9586-7836d5b6de15 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:38.310593 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-592811bf-8692-45dd-9586-7836d5b6de15 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:38.312045 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-592811bf-8692-45dd-9586-7836d5b6de15 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:38.312994 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-592811bf-8692-45dd-9586-7836d5b6de15 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:38.313746 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-592811bf-8692-45dd-9586-7836d5b6de15 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:38.506918 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-592811bf-8692-45dd-9586-7836d5b6de15 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:38.516188 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-592811bf-8692-45dd-9586-7836d5b6de15 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:38.517673 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-592811bf-8692-45dd-9586-7836d5b6de15 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:38.518904 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-592811bf-8692-45dd-9586-7836d5b6de15 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1744 microversion: 2.20 time: 0.235752 Oct 30 01:00:38.519846 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 271/548] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:00:38 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1744 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:00:38.997037 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-ea746551-8787-4d0b-a160-5393006173d0 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:38.997666 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-ea746551-8787-4d0b-a160-5393006173d0 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: c7be44ac-3b42-4ace-ac56-65c117e839c6] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:39.008331 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ea746551-8787-4d0b-a160-5393006173d0 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:39.009079 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ea746551-8787-4d0b-a160-5393006173d0 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:39.009890 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ea746551-8787-4d0b-a160-5393006173d0 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:39.010813 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ea746551-8787-4d0b-a160-5393006173d0 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:39.011427 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ea746551-8787-4d0b-a160-5393006173d0 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:39.012078 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ea746551-8787-4d0b-a160-5393006173d0 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:39.080519 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-1a0e4295-d334-4038-9618-e0000a06a163 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:39.082170 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-1a0e4295-d334-4038-9618-e0000a06a163 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: db9ac414-31a5-4570-aa3f-37db3dac2049] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:39.099935 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1a0e4295-d334-4038-9618-e0000a06a163 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:39.107249 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1a0e4295-d334-4038-9618-e0000a06a163 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:39.108540 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1a0e4295-d334-4038-9618-e0000a06a163 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:39.112904 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1a0e4295-d334-4038-9618-e0000a06a163 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:39.118653 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1a0e4295-d334-4038-9618-e0000a06a163 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:39.119547 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1a0e4295-d334-4038-9618-e0000a06a163 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:39.263423 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-ea746551-8787-4d0b-a160-5393006173d0 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:39.272294 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-ea746551-8787-4d0b-a160-5393006173d0 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6" status: 200 len: 1502 microversion: 2.1 time: 0.279606 Oct 30 01:00:39.284181 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 278/549] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:38 2022] GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6 => generated 1502 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:39.292329 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [req-fa95b529-5ea2-483a-96ea-10dcf975a3da req-6f6e6849-1829-4db3-ac69-b5009c3d475b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9491a8f4-2b76-4a4e-a6bd-6720114e482f", "name": "network-vif-plugged", "status": "completed", "tag": "5367850e-73d0-4814-af96-bd430c94c272"}]} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:00:39.325269 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-fa95b529-5ea2-483a-96ea-10dcf975a3da req-6f6e6849-1829-4db3-ac69-b5009c3d475b service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:39.325904 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-fa95b529-5ea2-483a-96ea-10dcf975a3da req-6f6e6849-1829-4db3-ac69-b5009c3d475b service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:39.327609 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-fa95b529-5ea2-483a-96ea-10dcf975a3da req-6f6e6849-1829-4db3-ac69-b5009c3d475b service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:39.353874 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.compute.server_external_events [req-fa95b529-5ea2-483a-96ea-10dcf975a3da req-6f6e6849-1829-4db3-ac69-b5009c3d475b service nova] Creating event network-vif-plugged:5367850e-73d0-4814-af96-bd430c94c272 for instance 9491a8f4-2b76-4a4e-a6bd-6720114e482f on hv-cinder-maste Oct 30 01:00:39.360110 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [req-fa95b529-5ea2-483a-96ea-10dcf975a3da req-6f6e6849-1829-4db3-ac69-b5009c3d475b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.074370 Oct 30 01:00:39.360814 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 279/550] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:00:39 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:00:39.366665 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-3e1dfb14-500f-480c-9be8-684281b0d911 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "45dc8eb1-424a-4d96-b8ef-3c4e47f2e143"}} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:00:39.373959 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-3e1dfb14-500f-480c-9be8-684281b0d911 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: c7be44ac-3b42-4ace-ac56-65c117e839c6] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:39.380908 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3e1dfb14-500f-480c-9be8-684281b0d911 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:39.381450 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3e1dfb14-500f-480c-9be8-684281b0d911 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:39.381981 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3e1dfb14-500f-480c-9be8-684281b0d911 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:39.382789 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3e1dfb14-500f-480c-9be8-684281b0d911 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:39.383342 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3e1dfb14-500f-480c-9be8-684281b0d911 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:39.383951 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3e1dfb14-500f-480c-9be8-684281b0d911 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:39.392989 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-1a0e4295-d334-4038-9618-e0000a06a163 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:39.402314 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-1a0e4295-d334-4038-9618-e0000a06a163 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049" status: 200 len: 1319 microversion: 2.1 time: 0.327164 Oct 30 01:00:39.409911 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.wsgi [None req-3e1dfb14-500f-480c-9be8-684281b0d911 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] HTTP exception thrown: Cannot 'attach_volume' instance c7be44ac-3b42-4ace-ac56-65c117e839c6 while it is in vm_state rescued Oct 30 01:00:39.412943 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-3e1dfb14-500f-480c-9be8-684281b0d911 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Request method failure captured: Oct 30 01:00:39.412943 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: request: POST /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6/os-volume_attachments HTTP/1.1 Oct 30 01:00:39.412943 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Accept: application/json Oct 30 01:00:39.412943 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Accept-Encoding: identity Oct 30 01:00:39.412943 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Connection: close Oct 30 01:00:39.412943 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Content-Length: 74 Oct 30 01:00:39.412943 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Content-Type: application/json Oct 30 01:00:39.412943 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Host: 10.222.0.16 Oct 30 01:00:39.412943 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Openstack-System-Scope: None Oct 30 01:00:39.412943 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: User-Agent: python-urllib3/1.26.12 Oct 30 01:00:39.412943 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Auth-Token: gAAAAABjXcwtPy0eLS3MC3-cmfmCjFSfeavovRIYXfEfNVAywVbIQ30ctEe5V4sGvi1A3M4sDeUVrGvF9i6stAWwO3TejIedmS8RLhBtx-h_JsINfLnyH884rCMd4-euC7isoMsjNUbUBUpUbdHwx6JXx5oDJcwXMr1RZh9X14AJM7k_e6z6bu8 Oct 30 01:00:39.412943 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Domain-Id: None Oct 30 01:00:39.412943 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Domain-Name: None Oct 30 01:00:39.412943 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Identity-Status: Confirmed Oct 30 01:00:39.412943 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Is-Admin-Project: True Oct 30 01:00:39.412943 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Domain-Id: default Oct 30 01:00:39.412943 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Domain-Name: Default Oct 30 01:00:39.412943 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Id: f48276ac51dd4bb1912fc670433cf6c4 Oct 30 01:00:39.412943 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-764720217 Oct 30 01:00:39.412943 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Role: reader,member Oct 30 01:00:39.412943 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Roles: reader,member Oct 30 01:00:39.414233 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_f48276ac51dd4bb1912fc670433cf6c4"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f48276ac51dd4bb1912fc670433cf6c4"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/f48276ac51dd4bb1912fc670433cf6c4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f48276ac51dd4bb1912fc670433cf6c4"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 01:00:39.414233 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant: tempest-ServerRescueNegativeTestJSON-764720217 Oct 30 01:00:39.414233 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant-Id: f48276ac51dd4bb1912fc670433cf6c4 Oct 30 01:00:39.414233 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-764720217 Oct 30 01:00:39.414233 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User: tempest-ServerRescueNegativeTestJSON-764720217-project Oct 30 01:00:39.414233 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Domain-Id: default Oct 30 01:00:39.414233 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Domain-Name: Default Oct 30 01:00:39.414233 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Id: 31b7342ecf07408cab6bb040f361fa1f Oct 30 01:00:39.414233 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Name: tempest-ServerRescueNegativeTestJSON-764720217-project Oct 30 01:00:39.414233 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Oct 30 01:00:39.414233 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: {"volumeAttachment": {"volumeId": "45dc8eb1-424a-4d96-b8ef-3c4e47f2e143"}} Oct 30 01:00:39.414233 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: method: > Oct 30 01:00:39.414982 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: exception: Cannot 'attach_volume' instance c7be44ac-3b42-4ace-ac56-65c117e839c6 while it is in vm_state rescued Oct 30 01:00:39.414982 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: action_args: {'server_id': 'c7be44ac-3b42-4ace-ac56-65c117e839c6', 'body': {'volumeAttachment': {'volumeId': '45dc8eb1-424a-4d96-b8ef-3c4e47f2e143'}}} Oct 30 01:00:39.414982 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 01:00:39.414982 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:00:39.414982 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Oct 30 01:00:39.414982 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Oct 30 01:00:39.414982 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 30 01:00:39.414982 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 30 01:00:39.414982 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Oct 30 01:00:39.414982 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Oct 30 01:00:39.414982 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance c7be44ac-3b42-4ace-ac56-65c117e839c6 in vm_state rescued. Cannot attach_volume while the instance is in this state. Oct 30 01:00:39.414982 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:00:39.414982 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:00:39.414982 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:00:39.414982 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:00:39.414982 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:00:39.414982 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:00:39.414982 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 01:00:39.414982 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 01:00:39.416269 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 01:00:39.416269 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 01:00:39.416269 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 30 01:00:39.416269 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 30 01:00:39.416269 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 30 01:00:39.416269 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 30 01:00:39.416269 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 30 01:00:39.416269 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 30 01:00:39.416269 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 398, in create Oct 30 01:00:39.416269 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Oct 30 01:00:39.416269 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Oct 30 01:00:39.416269 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Oct 30 01:00:39.416269 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'attach_volume' instance c7be44ac-3b42-4ace-ac56-65c117e839c6 while it is in vm_state rescued Oct 30 01:00:39.416269 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:00:39.416269 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:00:39.416269 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:00:39.416269 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:00:39.416269 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:00:39.416269 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:00:39.417370 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 01:00:39.417370 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 01:00:39.417370 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'attach_volume' instance c7be44ac-3b42-4ace-ac56-65c117e839c6 while it is in vm_state rescued Oct 30 01:00:39.417370 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:00:39.418035 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 272/551] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:39 2022] GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049 => generated 1319 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 01:00:39.418815 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-3e1dfb14-500f-480c-9be8-684281b0d911 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Returning 409 to user: Cannot 'attach_volume' instance c7be44ac-3b42-4ace-ac56-65c117e839c6 while it is in vm_state rescued {{(pid=118961) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 01:00:39.419667 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-3e1dfb14-500f-480c-9be8-684281b0d911 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "POST /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6/os-volume_attachments" status: 409 len: 152 microversion: 2.1 time: 0.057174 Oct 30 01:00:39.420277 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 280/552] 10.222.0.16 () {60 vars in 1323 bytes} [Sun Oct 30 01:00:39 2022] POST /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6/os-volume_attachments => generated 152 bytes in 59 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Oct 30 01:00:39.429214 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-7684365d-8bb1-43c7-89ac-0f4202dfdc2c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:00:39.429821 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-7684365d-8bb1-43c7-89ac-0f4202dfdc2c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: c7be44ac-3b42-4ace-ac56-65c117e839c6] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:39.437053 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7684365d-8bb1-43c7-89ac-0f4202dfdc2c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:39.437626 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7684365d-8bb1-43c7-89ac-0f4202dfdc2c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:39.438123 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7684365d-8bb1-43c7-89ac-0f4202dfdc2c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:39.438889 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7684365d-8bb1-43c7-89ac-0f4202dfdc2c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:39.439414 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7684365d-8bb1-43c7-89ac-0f4202dfdc2c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:39.439948 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7684365d-8bb1-43c7-89ac-0f4202dfdc2c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:39.527445 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-7684365d-8bb1-43c7-89ac-0f4202dfdc2c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "POST /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6/action" status: 202 len: 0 microversion: 2.1 time: 0.101762 Oct 30 01:00:39.528188 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 273/553] 10.222.0.16 () {60 vars in 1278 bytes} [Sun Oct 30 01:00:39 2022] POST /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6/action => generated 0 bytes in 103 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 30 01:00:39.536741 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-eac32a67-2a21-4075-8a11-2ab755728162 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:39.537643 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-eac32a67-2a21-4075-8a11-2ab755728162 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: c7be44ac-3b42-4ace-ac56-65c117e839c6] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:39.544556 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-eac32a67-2a21-4075-8a11-2ab755728162 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:39.545489 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-eac32a67-2a21-4075-8a11-2ab755728162 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:39.546313 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-eac32a67-2a21-4075-8a11-2ab755728162 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:39.547229 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-eac32a67-2a21-4075-8a11-2ab755728162 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:39.548090 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-eac32a67-2a21-4075-8a11-2ab755728162 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:39.548729 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-eac32a67-2a21-4075-8a11-2ab755728162 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:39.557995 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-9b9da3a3-daff-4973-9ff3-6a52f693c3cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:39.558711 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-9b9da3a3-daff-4973-9ff3-6a52f693c3cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:39.569815 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9b9da3a3-daff-4973-9ff3-6a52f693c3cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:39.571014 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9b9da3a3-daff-4973-9ff3-6a52f693c3cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:39.572982 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9b9da3a3-daff-4973-9ff3-6a52f693c3cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:39.574279 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9b9da3a3-daff-4973-9ff3-6a52f693c3cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:39.575056 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9b9da3a3-daff-4973-9ff3-6a52f693c3cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:39.575650 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9b9da3a3-daff-4973-9ff3-6a52f693c3cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:39.793254 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-eac32a67-2a21-4075-8a11-2ab755728162 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:39.805390 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-eac32a67-2a21-4075-8a11-2ab755728162 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6" status: 200 len: 1510 microversion: 2.1 time: 0.272650 Oct 30 01:00:39.805987 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 281/554] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:39 2022] GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6 => generated 1510 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:39.818860 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-81ab6ca9-2489-42b6-8599-095b4d966781 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:00:39.843135 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-81ab6ca9-2489-42b6-8599-095b4d966781 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:39.846187 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-81ab6ca9-2489-42b6-8599-095b4d966781 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:39.848286 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-81ab6ca9-2489-42b6-8599-095b4d966781 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:39.874528 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-9b9da3a3-daff-4973-9ff3-6a52f693c3cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:39.884869 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-9b9da3a3-daff-4973-9ff3-6a52f693c3cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:39.886890 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-9b9da3a3-daff-4973-9ff3-6a52f693c3cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:39.888492 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-9b9da3a3-daff-4973-9ff3-6a52f693c3cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1744 microversion: 2.20 time: 0.347463 Oct 30 01:00:39.890690 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 274/555] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:00:39 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1744 bytes in 351 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:00:39.977521 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-81ab6ca9-2489-42b6-8599-095b4d966781 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Going to try to stop instance {{(pid=118961) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Oct 30 01:00:40.061345 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-81ab6ca9-2489-42b6-8599-095b4d966781 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "POST /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14/action" status: 202 len: 0 microversion: 2.1 time: 0.253352 Oct 30 01:00:40.062134 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 282/556] 10.222.0.16 () {60 vars in 1278 bytes} [Sun Oct 30 01:00:39 2022] POST /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14/action => generated 0 bytes in 255 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 30 01:00:40.071298 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-35a11332-ca57-4d3d-927c-1948520d7572 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:40.071970 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-35a11332-ca57-4d3d-927c-1948520d7572 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:40.079629 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-35a11332-ca57-4d3d-927c-1948520d7572 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:40.080164 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-35a11332-ca57-4d3d-927c-1948520d7572 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:40.080613 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-35a11332-ca57-4d3d-927c-1948520d7572 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:40.084082 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-35a11332-ca57-4d3d-927c-1948520d7572 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:40.084586 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-35a11332-ca57-4d3d-927c-1948520d7572 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:40.085007 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-35a11332-ca57-4d3d-927c-1948520d7572 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:40.257149 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-35a11332-ca57-4d3d-927c-1948520d7572 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:40.266717 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-35a11332-ca57-4d3d-927c-1948520d7572 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1724 microversion: 2.1 time: 0.199312 Oct 30 01:00:40.268153 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 275/557] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:40 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1724 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:40.428186 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-f8990363-fb69-4958-a1c8-db3ed0a6bfa4 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:40.429404 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-f8990363-fb69-4958-a1c8-db3ed0a6bfa4 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: db9ac414-31a5-4570-aa3f-37db3dac2049] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:40.431462 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell0:SELECT=2 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:00:40.443645 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f8990363-fb69-4958-a1c8-db3ed0a6bfa4 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:40.444187 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f8990363-fb69-4958-a1c8-db3ed0a6bfa4 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:40.444696 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f8990363-fb69-4958-a1c8-db3ed0a6bfa4 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:40.445380 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f8990363-fb69-4958-a1c8-db3ed0a6bfa4 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:40.445830 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f8990363-fb69-4958-a1c8-db3ed0a6bfa4 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:40.446258 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f8990363-fb69-4958-a1c8-db3ed0a6bfa4 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:40.609345 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-f8990363-fb69-4958-a1c8-db3ed0a6bfa4 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:40.617434 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-f8990363-fb69-4958-a1c8-db3ed0a6bfa4 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049" status: 200 len: 1319 microversion: 2.1 time: 0.195176 Oct 30 01:00:40.618601 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 283/558] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:40 2022] GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049 => generated 1319 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:40.830994 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-dca1626e-7f6f-462d-bc01-f12a8f6c0a51 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:40.831629 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-dca1626e-7f6f-462d-bc01-f12a8f6c0a51 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: c7be44ac-3b42-4ace-ac56-65c117e839c6] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:40.839096 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dca1626e-7f6f-462d-bc01-f12a8f6c0a51 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:40.839680 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dca1626e-7f6f-462d-bc01-f12a8f6c0a51 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:40.840415 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dca1626e-7f6f-462d-bc01-f12a8f6c0a51 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:40.841393 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dca1626e-7f6f-462d-bc01-f12a8f6c0a51 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:40.842346 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dca1626e-7f6f-462d-bc01-f12a8f6c0a51 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:40.843040 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dca1626e-7f6f-462d-bc01-f12a8f6c0a51 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:40.919240 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-172902fc-e5ba-4771-b3db-bc486f66de70 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:40.920173 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-172902fc-e5ba-4771-b3db-bc486f66de70 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:40.925384 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell0:SELECT=8 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:00:40.927518 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-172902fc-e5ba-4771-b3db-bc486f66de70 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:40.928279 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-172902fc-e5ba-4771-b3db-bc486f66de70 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:40.929034 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-172902fc-e5ba-4771-b3db-bc486f66de70 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:40.929902 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-172902fc-e5ba-4771-b3db-bc486f66de70 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:40.930506 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-172902fc-e5ba-4771-b3db-bc486f66de70 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:40.931079 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-172902fc-e5ba-4771-b3db-bc486f66de70 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:41.064901 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-dca1626e-7f6f-462d-bc01-f12a8f6c0a51 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:41.076186 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-dca1626e-7f6f-462d-bc01-f12a8f6c0a51 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6" status: 200 len: 1510 microversion: 2.1 time: 0.249618 Oct 30 01:00:41.082739 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 276/559] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:40 2022] GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6 => generated 1510 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:41.138911 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-172902fc-e5ba-4771-b3db-bc486f66de70 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:41.149450 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-172902fc-e5ba-4771-b3db-bc486f66de70 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:41.151411 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-172902fc-e5ba-4771-b3db-bc486f66de70 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:41.153576 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-172902fc-e5ba-4771-b3db-bc486f66de70 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1744 microversion: 2.20 time: 0.238402 Oct 30 01:00:41.154232 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 284/560] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:00:40 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1744 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:00:41.302075 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-af1f58ae-9313-4843-9dae-b99c3dadd3d6 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:41.302075 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-af1f58ae-9313-4843-9dae-b99c3dadd3d6 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:41.309099 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-af1f58ae-9313-4843-9dae-b99c3dadd3d6 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:41.309735 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-af1f58ae-9313-4843-9dae-b99c3dadd3d6 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:41.310287 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-af1f58ae-9313-4843-9dae-b99c3dadd3d6 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:41.311086 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-af1f58ae-9313-4843-9dae-b99c3dadd3d6 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:41.311604 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-af1f58ae-9313-4843-9dae-b99c3dadd3d6 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:41.312127 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-af1f58ae-9313-4843-9dae-b99c3dadd3d6 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:41.476000 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-af1f58ae-9313-4843-9dae-b99c3dadd3d6 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:41.485389 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-af1f58ae-9313-4843-9dae-b99c3dadd3d6 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1724 microversion: 2.1 time: 0.190310 Oct 30 01:00:41.486493 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 277/561] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:41 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1724 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:41.640840 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-dd66887b-f79a-4ddf-84b6-e155a8036205 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:41.642770 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-dd66887b-f79a-4ddf-84b6-e155a8036205 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: db9ac414-31a5-4570-aa3f-37db3dac2049] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:41.650903 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-dd66887b-f79a-4ddf-84b6-e155a8036205 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:41.651664 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-dd66887b-f79a-4ddf-84b6-e155a8036205 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:41.652273 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-dd66887b-f79a-4ddf-84b6-e155a8036205 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:41.653215 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-dd66887b-f79a-4ddf-84b6-e155a8036205 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:41.653704 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-dd66887b-f79a-4ddf-84b6-e155a8036205 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:41.654410 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-dd66887b-f79a-4ddf-84b6-e155a8036205 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:41.829008 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-dd66887b-f79a-4ddf-84b6-e155a8036205 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:41.837907 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-dd66887b-f79a-4ddf-84b6-e155a8036205 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049" status: 200 len: 1319 microversion: 2.1 time: 0.200880 Oct 30 01:00:41.838761 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 285/562] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:41 2022] GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049 => generated 1319 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:42.102268 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-325e7861-0ba8-433d-bee5-8da5bdee2bd8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:42.102944 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-325e7861-0ba8-433d-bee5-8da5bdee2bd8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: c7be44ac-3b42-4ace-ac56-65c117e839c6] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:42.111639 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-325e7861-0ba8-433d-bee5-8da5bdee2bd8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:42.112159 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-325e7861-0ba8-433d-bee5-8da5bdee2bd8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:42.112594 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-325e7861-0ba8-433d-bee5-8da5bdee2bd8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:42.113297 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-325e7861-0ba8-433d-bee5-8da5bdee2bd8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:42.113739 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-325e7861-0ba8-433d-bee5-8da5bdee2bd8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:42.114151 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-325e7861-0ba8-433d-bee5-8da5bdee2bd8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:42.179067 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-14d0885f-548e-46e4-ac91-ad695cb2582a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:42.179855 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-14d0885f-548e-46e4-ac91-ad695cb2582a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:42.187578 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-14d0885f-548e-46e4-ac91-ad695cb2582a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:42.188174 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-14d0885f-548e-46e4-ac91-ad695cb2582a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:42.188687 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-14d0885f-548e-46e4-ac91-ad695cb2582a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:42.189494 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-14d0885f-548e-46e4-ac91-ad695cb2582a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:42.190007 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-14d0885f-548e-46e4-ac91-ad695cb2582a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:42.190485 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-14d0885f-548e-46e4-ac91-ad695cb2582a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:42.301766 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-325e7861-0ba8-433d-bee5-8da5bdee2bd8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:42.313579 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-325e7861-0ba8-433d-bee5-8da5bdee2bd8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6" status: 200 len: 1510 microversion: 2.1 time: 0.215666 Oct 30 01:00:42.314404 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 278/563] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:42 2022] GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6 => generated 1510 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:42.352516 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-14d0885f-548e-46e4-ac91-ad695cb2582a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:42.361685 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-14d0885f-548e-46e4-ac91-ad695cb2582a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:42.363112 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-14d0885f-548e-46e4-ac91-ad695cb2582a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:42.364317 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-14d0885f-548e-46e4-ac91-ad695cb2582a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1744 microversion: 2.20 time: 0.189326 Oct 30 01:00:42.364903 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 286/564] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:00:42 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1744 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:00:42.520320 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-1925b595-9f64-4e9c-820d-6582d337b776 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:42.521392 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-1925b595-9f64-4e9c-820d-6582d337b776 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:42.530721 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1925b595-9f64-4e9c-820d-6582d337b776 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:42.531209 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1925b595-9f64-4e9c-820d-6582d337b776 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:42.531918 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1925b595-9f64-4e9c-820d-6582d337b776 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:42.532348 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1925b595-9f64-4e9c-820d-6582d337b776 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:42.532812 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1925b595-9f64-4e9c-820d-6582d337b776 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:42.533205 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1925b595-9f64-4e9c-820d-6582d337b776 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:42.673905 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-1925b595-9f64-4e9c-820d-6582d337b776 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:42.682460 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-1925b595-9f64-4e9c-820d-6582d337b776 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1724 microversion: 2.1 time: 0.168873 Oct 30 01:00:42.683030 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 279/565] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:42 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1724 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:42.866893 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-a3e05926-2091-4113-b414-9e46cffa80c4 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:42.867510 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-a3e05926-2091-4113-b414-9e46cffa80c4 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: db9ac414-31a5-4570-aa3f-37db3dac2049] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:42.873988 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a3e05926-2091-4113-b414-9e46cffa80c4 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:42.874480 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a3e05926-2091-4113-b414-9e46cffa80c4 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:42.874913 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a3e05926-2091-4113-b414-9e46cffa80c4 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:42.875984 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a3e05926-2091-4113-b414-9e46cffa80c4 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:42.876090 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a3e05926-2091-4113-b414-9e46cffa80c4 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:42.876503 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a3e05926-2091-4113-b414-9e46cffa80c4 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:43.011178 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-a3e05926-2091-4113-b414-9e46cffa80c4 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:43.019406 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-a3e05926-2091-4113-b414-9e46cffa80c4 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049" status: 200 len: 1319 microversion: 2.1 time: 0.156708 Oct 30 01:00:43.020069 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 287/566] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:42 2022] GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049 => generated 1319 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:43.336941 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-696f4e3e-51ee-4906-b084-0b0c232dd8cf tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:43.337554 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-696f4e3e-51ee-4906-b084-0b0c232dd8cf tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: c7be44ac-3b42-4ace-ac56-65c117e839c6] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:43.345432 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-696f4e3e-51ee-4906-b084-0b0c232dd8cf tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:43.346146 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-696f4e3e-51ee-4906-b084-0b0c232dd8cf tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:43.346575 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-696f4e3e-51ee-4906-b084-0b0c232dd8cf tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:43.347271 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-696f4e3e-51ee-4906-b084-0b0c232dd8cf tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:43.347745 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-696f4e3e-51ee-4906-b084-0b0c232dd8cf tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:43.348149 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-696f4e3e-51ee-4906-b084-0b0c232dd8cf tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:43.390412 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-076df240-f757-452f-b9df-7c16b758cd5a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:43.391028 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-076df240-f757-452f-b9df-7c16b758cd5a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:43.413870 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-076df240-f757-452f-b9df-7c16b758cd5a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:43.414396 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-076df240-f757-452f-b9df-7c16b758cd5a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:43.414850 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-076df240-f757-452f-b9df-7c16b758cd5a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:43.415852 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-076df240-f757-452f-b9df-7c16b758cd5a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:43.416479 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-076df240-f757-452f-b9df-7c16b758cd5a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:43.416923 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-076df240-f757-452f-b9df-7c16b758cd5a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:43.533030 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-696f4e3e-51ee-4906-b084-0b0c232dd8cf tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:43.541286 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-696f4e3e-51ee-4906-b084-0b0c232dd8cf tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6" status: 200 len: 1510 microversion: 2.1 time: 0.208350 Oct 30 01:00:43.542234 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 280/567] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:43 2022] GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6 => generated 1510 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:43.578860 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-076df240-f757-452f-b9df-7c16b758cd5a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:43.587198 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-076df240-f757-452f-b9df-7c16b758cd5a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:43.588859 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-076df240-f757-452f-b9df-7c16b758cd5a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:43.590009 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-076df240-f757-452f-b9df-7c16b758cd5a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1744 microversion: 2.20 time: 0.203578 Oct 30 01:00:43.598517 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 288/568] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:00:43 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1744 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:00:43.709081 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-e758c124-26a4-430a-a7cd-0f5cc616670b tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:43.709874 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-e758c124-26a4-430a-a7cd-0f5cc616670b tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:43.718582 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e758c124-26a4-430a-a7cd-0f5cc616670b tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:43.719605 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e758c124-26a4-430a-a7cd-0f5cc616670b tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:43.719605 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e758c124-26a4-430a-a7cd-0f5cc616670b tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:43.720178 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e758c124-26a4-430a-a7cd-0f5cc616670b tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:43.720621 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e758c124-26a4-430a-a7cd-0f5cc616670b tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:43.721029 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e758c124-26a4-430a-a7cd-0f5cc616670b tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:43.860344 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-e758c124-26a4-430a-a7cd-0f5cc616670b tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:43.870782 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-e758c124-26a4-430a-a7cd-0f5cc616670b tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1724 microversion: 2.1 time: 0.167648 Oct 30 01:00:43.871393 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 281/569] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:43 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1724 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:44.046813 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-80b75e3e-94f3-4d25-80c6-888a9870d086 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:44.047879 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-80b75e3e-94f3-4d25-80c6-888a9870d086 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: db9ac414-31a5-4570-aa3f-37db3dac2049] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:44.057434 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-80b75e3e-94f3-4d25-80c6-888a9870d086 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:44.057910 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-80b75e3e-94f3-4d25-80c6-888a9870d086 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:44.058337 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-80b75e3e-94f3-4d25-80c6-888a9870d086 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:44.059023 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-80b75e3e-94f3-4d25-80c6-888a9870d086 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:44.059466 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-80b75e3e-94f3-4d25-80c6-888a9870d086 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:44.059912 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-80b75e3e-94f3-4d25-80c6-888a9870d086 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:44.204717 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-80b75e3e-94f3-4d25-80c6-888a9870d086 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:44.213542 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-80b75e3e-94f3-4d25-80c6-888a9870d086 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049" status: 200 len: 1319 microversion: 2.1 time: 0.173400 Oct 30 01:00:44.214124 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 289/570] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:44 2022] GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049 => generated 1319 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:44.566610 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-0b6df018-3e56-4980-9e95-263108f7686c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:44.567328 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-0b6df018-3e56-4980-9e95-263108f7686c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: c7be44ac-3b42-4ace-ac56-65c117e839c6] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:44.569962 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_api:SELECT=84 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:00:44.575495 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0b6df018-3e56-4980-9e95-263108f7686c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:44.576121 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0b6df018-3e56-4980-9e95-263108f7686c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:44.576625 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0b6df018-3e56-4980-9e95-263108f7686c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:44.577708 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0b6df018-3e56-4980-9e95-263108f7686c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:44.578235 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0b6df018-3e56-4980-9e95-263108f7686c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:44.578724 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0b6df018-3e56-4980-9e95-263108f7686c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:44.614657 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-bbbddff8-4812-4119-a9f1-ef2b44876607 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:44.615270 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-bbbddff8-4812-4119-a9f1-ef2b44876607 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:44.621805 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bbbddff8-4812-4119-a9f1-ef2b44876607 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:44.622293 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bbbddff8-4812-4119-a9f1-ef2b44876607 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:44.622744 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bbbddff8-4812-4119-a9f1-ef2b44876607 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:44.623458 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bbbddff8-4812-4119-a9f1-ef2b44876607 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:44.624111 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bbbddff8-4812-4119-a9f1-ef2b44876607 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:44.624538 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bbbddff8-4812-4119-a9f1-ef2b44876607 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:44.824069 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-0b6df018-3e56-4980-9e95-263108f7686c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:44.832553 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-0b6df018-3e56-4980-9e95-263108f7686c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6" status: 200 len: 1510 microversion: 2.1 time: 0.269570 Oct 30 01:00:44.833568 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 282/571] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:44 2022] GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6 => generated 1510 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:44.864670 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-bbbddff8-4812-4119-a9f1-ef2b44876607 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:44.872812 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-bbbddff8-4812-4119-a9f1-ef2b44876607 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:44.875939 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-bbbddff8-4812-4119-a9f1-ef2b44876607 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:44.877119 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-bbbddff8-4812-4119-a9f1-ef2b44876607 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1744 microversion: 2.20 time: 0.266557 Oct 30 01:00:44.877685 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 290/572] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:00:44 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1744 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:00:44.898955 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-8336c62a-e831-4b9e-89e1-335b75471ffa tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:44.899398 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-8336c62a-e831-4b9e-89e1-335b75471ffa tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:44.906077 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8336c62a-e831-4b9e-89e1-335b75471ffa tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:44.906632 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8336c62a-e831-4b9e-89e1-335b75471ffa tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:44.907097 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8336c62a-e831-4b9e-89e1-335b75471ffa tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:44.908049 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8336c62a-e831-4b9e-89e1-335b75471ffa tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:44.909151 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8336c62a-e831-4b9e-89e1-335b75471ffa tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:44.909568 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8336c62a-e831-4b9e-89e1-335b75471ffa tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:45.050912 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-8336c62a-e831-4b9e-89e1-335b75471ffa tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:45.060673 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-8336c62a-e831-4b9e-89e1-335b75471ffa tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1724 microversion: 2.1 time: 0.165953 Oct 30 01:00:45.061248 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 283/573] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:44 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1724 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:45.236626 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-7531eb80-c421-401c-81cf-71e30ede0da2 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:45.237325 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-7531eb80-c421-401c-81cf-71e30ede0da2 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: db9ac414-31a5-4570-aa3f-37db3dac2049] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:45.243844 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7531eb80-c421-401c-81cf-71e30ede0da2 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:45.244540 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7531eb80-c421-401c-81cf-71e30ede0da2 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:45.245240 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7531eb80-c421-401c-81cf-71e30ede0da2 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:45.246141 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7531eb80-c421-401c-81cf-71e30ede0da2 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:45.246658 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7531eb80-c421-401c-81cf-71e30ede0da2 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:45.247107 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7531eb80-c421-401c-81cf-71e30ede0da2 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:45.419243 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-7531eb80-c421-401c-81cf-71e30ede0da2 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:45.427257 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-7531eb80-c421-401c-81cf-71e30ede0da2 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049" status: 200 len: 1319 microversion: 2.1 time: 0.194404 Oct 30 01:00:45.427851 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 291/574] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:45 2022] GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049 => generated 1319 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:45.855500 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-801e7629-bb83-419f-b73d-5109d6ff13cb tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:45.856161 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-801e7629-bb83-419f-b73d-5109d6ff13cb tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: c7be44ac-3b42-4ace-ac56-65c117e839c6] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:45.862723 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-801e7629-bb83-419f-b73d-5109d6ff13cb tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:45.863238 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-801e7629-bb83-419f-b73d-5109d6ff13cb tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:45.863663 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-801e7629-bb83-419f-b73d-5109d6ff13cb tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:45.864375 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-801e7629-bb83-419f-b73d-5109d6ff13cb tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:45.864801 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-801e7629-bb83-419f-b73d-5109d6ff13cb tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:45.865211 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-801e7629-bb83-419f-b73d-5109d6ff13cb tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:45.903551 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-4fb3e1ad-0983-4bad-a18c-f96c05e48000 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:45.904895 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-4fb3e1ad-0983-4bad-a18c-f96c05e48000 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:45.912229 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4fb3e1ad-0983-4bad-a18c-f96c05e48000 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:45.913014 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4fb3e1ad-0983-4bad-a18c-f96c05e48000 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:45.913646 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4fb3e1ad-0983-4bad-a18c-f96c05e48000 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:45.915194 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4fb3e1ad-0983-4bad-a18c-f96c05e48000 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:45.916216 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4fb3e1ad-0983-4bad-a18c-f96c05e48000 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:45.916722 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4fb3e1ad-0983-4bad-a18c-f96c05e48000 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:46.057984 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-801e7629-bb83-419f-b73d-5109d6ff13cb tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:46.074963 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-801e7629-bb83-419f-b73d-5109d6ff13cb tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6" status: 200 len: 1510 microversion: 2.1 time: 0.223371 Oct 30 01:00:46.076029 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 284/575] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:45 2022] GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6 => generated 1510 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:46.085791 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-4fb3e1ad-0983-4bad-a18c-f96c05e48000 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:46.091323 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-8c343a73-22aa-4ed4-8cd7-8df876a6d884 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:46.092072 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-8c343a73-22aa-4ed4-8cd7-8df876a6d884 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:46.097053 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-4fb3e1ad-0983-4bad-a18c-f96c05e48000 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:46.098920 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-4fb3e1ad-0983-4bad-a18c-f96c05e48000 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:46.100130 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-4fb3e1ad-0983-4bad-a18c-f96c05e48000 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1744 microversion: 2.20 time: 0.201566 Oct 30 01:00:46.101650 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 292/576] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:00:45 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1744 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:00:46.115221 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8c343a73-22aa-4ed4-8cd7-8df876a6d884 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:46.115822 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8c343a73-22aa-4ed4-8cd7-8df876a6d884 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:46.120365 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8c343a73-22aa-4ed4-8cd7-8df876a6d884 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:46.123981 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8c343a73-22aa-4ed4-8cd7-8df876a6d884 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:46.124537 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8c343a73-22aa-4ed4-8cd7-8df876a6d884 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:46.125087 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8c343a73-22aa-4ed4-8cd7-8df876a6d884 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:46.269272 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-8c343a73-22aa-4ed4-8cd7-8df876a6d884 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:46.272634 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:00:46.278984 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-8c343a73-22aa-4ed4-8cd7-8df876a6d884 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1724 microversion: 2.1 time: 0.192317 Oct 30 01:00:46.279575 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 285/577] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:46 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1724 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 01:00:46.450579 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-928ee47b-fb8b-4ca1-9c48-91004d7ccc36 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:46.451349 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-928ee47b-fb8b-4ca1-9c48-91004d7ccc36 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: db9ac414-31a5-4570-aa3f-37db3dac2049] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:46.458167 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-928ee47b-fb8b-4ca1-9c48-91004d7ccc36 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:46.458643 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-928ee47b-fb8b-4ca1-9c48-91004d7ccc36 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:46.459783 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-928ee47b-fb8b-4ca1-9c48-91004d7ccc36 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:46.460495 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-928ee47b-fb8b-4ca1-9c48-91004d7ccc36 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:46.461000 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-928ee47b-fb8b-4ca1-9c48-91004d7ccc36 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:46.461424 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-928ee47b-fb8b-4ca1-9c48-91004d7ccc36 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:46.593137 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-928ee47b-fb8b-4ca1-9c48-91004d7ccc36 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:46.596551 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:00:46.601050 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-928ee47b-fb8b-4ca1-9c48-91004d7ccc36 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049" status: 200 len: 1319 microversion: 2.1 time: 0.154578 Oct 30 01:00:46.601611 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 293/578] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:46 2022] GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049 => generated 1319 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:47.100588 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-468a5288-425d-47a8-91c4-498deb774980 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:47.101141 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-468a5288-425d-47a8-91c4-498deb774980 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: c7be44ac-3b42-4ace-ac56-65c117e839c6] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:47.108874 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-468a5288-425d-47a8-91c4-498deb774980 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:47.109350 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-468a5288-425d-47a8-91c4-498deb774980 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:47.109781 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-468a5288-425d-47a8-91c4-498deb774980 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:47.110458 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-468a5288-425d-47a8-91c4-498deb774980 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:47.110900 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-468a5288-425d-47a8-91c4-498deb774980 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:47.111315 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-468a5288-425d-47a8-91c4-498deb774980 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:47.135828 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-c601dde8-7ecc-4222-9b9e-92ce33189522 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:47.136701 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-c601dde8-7ecc-4222-9b9e-92ce33189522 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:47.144045 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c601dde8-7ecc-4222-9b9e-92ce33189522 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:47.144615 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c601dde8-7ecc-4222-9b9e-92ce33189522 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:47.145207 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c601dde8-7ecc-4222-9b9e-92ce33189522 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:47.146015 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c601dde8-7ecc-4222-9b9e-92ce33189522 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:47.146579 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c601dde8-7ecc-4222-9b9e-92ce33189522 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:47.147004 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c601dde8-7ecc-4222-9b9e-92ce33189522 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:47.305502 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-468a5288-425d-47a8-91c4-498deb774980 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:47.314340 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-468a5288-425d-47a8-91c4-498deb774980 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6" status: 200 len: 1510 microversion: 2.1 time: 0.217720 Oct 30 01:00:47.315863 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 286/579] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:47 2022] GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6 => generated 1510 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:47.328516 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-68f87d11-d4a4-46d2-b63c-ebde79261f7d tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:47.329282 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-68f87d11-d4a4-46d2-b63c-ebde79261f7d tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:47.335410 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-68f87d11-d4a4-46d2-b63c-ebde79261f7d tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:47.336009 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-68f87d11-d4a4-46d2-b63c-ebde79261f7d tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:47.336512 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-68f87d11-d4a4-46d2-b63c-ebde79261f7d tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:47.337274 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-68f87d11-d4a4-46d2-b63c-ebde79261f7d tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:47.337858 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-68f87d11-d4a4-46d2-b63c-ebde79261f7d tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:47.338336 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-68f87d11-d4a4-46d2-b63c-ebde79261f7d tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:47.340480 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-c601dde8-7ecc-4222-9b9e-92ce33189522 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:47.349494 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-c601dde8-7ecc-4222-9b9e-92ce33189522 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:47.350471 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-c601dde8-7ecc-4222-9b9e-92ce33189522 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:47.351671 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-c601dde8-7ecc-4222-9b9e-92ce33189522 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1744 microversion: 2.20 time: 0.220577 Oct 30 01:00:47.352286 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 294/580] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:00:47 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1744 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:00:47.479155 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-68f87d11-d4a4-46d2-b63c-ebde79261f7d tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:47.489196 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-68f87d11-d4a4-46d2-b63c-ebde79261f7d tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1724 microversion: 2.1 time: 0.172132 Oct 30 01:00:47.489804 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 287/581] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:47 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1724 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:47.609735 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [req-e23b2082-4060-4f76-90d8-d3a9ae1364bf req-4cfe1842-04ef-471e-8c34-b79e44bddbc6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "db9ac414-31a5-4570-aa3f-37db3dac2049", "name": "network-vif-plugged", "status": "completed", "tag": "56471ebd-e8c3-4ce0-bd02-87518e117371"}]} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:00:47.621160 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-e23b2082-4060-4f76-90d8-d3a9ae1364bf req-4cfe1842-04ef-471e-8c34-b79e44bddbc6 service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:47.621835 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-e23b2082-4060-4f76-90d8-d3a9ae1364bf req-4cfe1842-04ef-471e-8c34-b79e44bddbc6 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:47.622270 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-e23b2082-4060-4f76-90d8-d3a9ae1364bf req-4cfe1842-04ef-471e-8c34-b79e44bddbc6 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:47.625246 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-214c0a81-3c76-45b2-86a7-2ea10a70c7dd tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:47.625874 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-214c0a81-3c76-45b2-86a7-2ea10a70c7dd tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: db9ac414-31a5-4570-aa3f-37db3dac2049] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:47.632032 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-214c0a81-3c76-45b2-86a7-2ea10a70c7dd tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:47.632516 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-214c0a81-3c76-45b2-86a7-2ea10a70c7dd tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:47.632947 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-214c0a81-3c76-45b2-86a7-2ea10a70c7dd tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:47.633639 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-214c0a81-3c76-45b2-86a7-2ea10a70c7dd tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:47.634090 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-214c0a81-3c76-45b2-86a7-2ea10a70c7dd tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:47.634502 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-214c0a81-3c76-45b2-86a7-2ea10a70c7dd tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:47.634635 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.compute.server_external_events [req-e23b2082-4060-4f76-90d8-d3a9ae1364bf req-4cfe1842-04ef-471e-8c34-b79e44bddbc6 service nova] Creating event network-vif-plugged:56471ebd-e8c3-4ce0-bd02-87518e117371 for instance db9ac414-31a5-4570-aa3f-37db3dac2049 on hv-cinder-maste Oct 30 01:00:47.640309 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [req-e23b2082-4060-4f76-90d8-d3a9ae1364bf req-4cfe1842-04ef-471e-8c34-b79e44bddbc6 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034385 Oct 30 01:00:47.640911 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 295/582] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:00:47 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:00:47.789949 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-214c0a81-3c76-45b2-86a7-2ea10a70c7dd tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:47.798214 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-214c0a81-3c76-45b2-86a7-2ea10a70c7dd tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049" status: 200 len: 1319 microversion: 2.1 time: 0.178584 Oct 30 01:00:47.798805 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 288/583] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:47 2022] GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049 => generated 1319 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:48.341876 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-b60be5f9-aa38-4439-944a-a7592ba89911 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:48.342661 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-b60be5f9-aa38-4439-944a-a7592ba89911 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: c7be44ac-3b42-4ace-ac56-65c117e839c6] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:48.350255 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b60be5f9-aa38-4439-944a-a7592ba89911 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:48.350821 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b60be5f9-aa38-4439-944a-a7592ba89911 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:48.351414 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b60be5f9-aa38-4439-944a-a7592ba89911 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:48.352520 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b60be5f9-aa38-4439-944a-a7592ba89911 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:48.353165 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b60be5f9-aa38-4439-944a-a7592ba89911 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:48.353717 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b60be5f9-aa38-4439-944a-a7592ba89911 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:48.378393 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-3e3d52ef-9940-438c-afdf-f71a1631fa4d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:48.378991 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-3e3d52ef-9940-438c-afdf-f71a1631fa4d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:48.386396 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3e3d52ef-9940-438c-afdf-f71a1631fa4d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:48.386862 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3e3d52ef-9940-438c-afdf-f71a1631fa4d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:48.387294 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3e3d52ef-9940-438c-afdf-f71a1631fa4d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:48.388087 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3e3d52ef-9940-438c-afdf-f71a1631fa4d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:48.388538 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3e3d52ef-9940-438c-afdf-f71a1631fa4d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:48.388956 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3e3d52ef-9940-438c-afdf-f71a1631fa4d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:48.524715 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-3e3d52ef-9940-438c-afdf-f71a1631fa4d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:48.532924 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-3e3d52ef-9940-438c-afdf-f71a1631fa4d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:48.534339 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-3e3d52ef-9940-438c-afdf-f71a1631fa4d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:48.535482 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-3e3d52ef-9940-438c-afdf-f71a1631fa4d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1744 microversion: 2.20 time: 0.161084 Oct 30 01:00:48.536071 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 289/584] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:00:48 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1744 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:00:48.540565 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-3882f5c4-db47-494a-adaa-524998117981 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:48.541162 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-3882f5c4-db47-494a-adaa-524998117981 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:48.547704 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3882f5c4-db47-494a-adaa-524998117981 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:48.548080 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3882f5c4-db47-494a-adaa-524998117981 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:48.548501 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3882f5c4-db47-494a-adaa-524998117981 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:48.549181 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3882f5c4-db47-494a-adaa-524998117981 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:48.549803 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3882f5c4-db47-494a-adaa-524998117981 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:48.550218 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3882f5c4-db47-494a-adaa-524998117981 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:48.730869 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-b60be5f9-aa38-4439-944a-a7592ba89911 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:48.735361 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-3882f5c4-db47-494a-adaa-524998117981 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:48.739133 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-b60be5f9-aa38-4439-944a-a7592ba89911 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6" status: 200 len: 1510 microversion: 2.1 time: 0.402736 Oct 30 01:00:48.739685 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 296/585] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:48 2022] GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6 => generated 1510 bytes in 404 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:48.744444 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-3882f5c4-db47-494a-adaa-524998117981 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1724 microversion: 2.1 time: 0.207361 Oct 30 01:00:48.745099 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 290/586] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:48 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1724 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:48.824312 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-18c7c3d2-028a-4ca2-96ca-a3a7a1f760f0 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:48.825010 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-18c7c3d2-028a-4ca2-96ca-a3a7a1f760f0 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: db9ac414-31a5-4570-aa3f-37db3dac2049] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:48.832147 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-18c7c3d2-028a-4ca2-96ca-a3a7a1f760f0 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:48.832635 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-18c7c3d2-028a-4ca2-96ca-a3a7a1f760f0 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:48.833074 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-18c7c3d2-028a-4ca2-96ca-a3a7a1f760f0 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:48.833777 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-18c7c3d2-028a-4ca2-96ca-a3a7a1f760f0 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:48.834234 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-18c7c3d2-028a-4ca2-96ca-a3a7a1f760f0 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:48.834660 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-18c7c3d2-028a-4ca2-96ca-a3a7a1f760f0 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:48.985736 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-18c7c3d2-028a-4ca2-96ca-a3a7a1f760f0 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:48.997293 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-18c7c3d2-028a-4ca2-96ca-a3a7a1f760f0 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049" status: 200 len: 1319 microversion: 2.1 time: 0.178614 Oct 30 01:00:48.998002 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 297/587] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:48 2022] GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049 => generated 1319 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:49.565444 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-0e4803b5-9192-4b3c-8603-fc4410999949 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:49.566372 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-0e4803b5-9192-4b3c-8603-fc4410999949 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:49.576185 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0e4803b5-9192-4b3c-8603-fc4410999949 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:49.576667 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0e4803b5-9192-4b3c-8603-fc4410999949 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:49.577434 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0e4803b5-9192-4b3c-8603-fc4410999949 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:49.578124 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0e4803b5-9192-4b3c-8603-fc4410999949 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:49.578573 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0e4803b5-9192-4b3c-8603-fc4410999949 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:49.578986 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0e4803b5-9192-4b3c-8603-fc4410999949 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:49.725937 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-0e4803b5-9192-4b3c-8603-fc4410999949 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:49.741511 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-0e4803b5-9192-4b3c-8603-fc4410999949 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:49.743242 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-0e4803b5-9192-4b3c-8603-fc4410999949 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:49.744462 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-0e4803b5-9192-4b3c-8603-fc4410999949 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1744 microversion: 2.20 time: 0.186074 Oct 30 01:00:49.745450 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 291/588] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:00:49 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1744 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:00:49.762715 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-943f0772-fca0-4869-b2c2-6ec65bbacf75 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:49.764089 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-943f0772-fca0-4869-b2c2-6ec65bbacf75 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: c7be44ac-3b42-4ace-ac56-65c117e839c6] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:49.767798 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-f2a7d76e-f2f6-4257-818b-b161a7195bde tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:49.768846 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-f2a7d76e-f2f6-4257-818b-b161a7195bde tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:49.771119 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-943f0772-fca0-4869-b2c2-6ec65bbacf75 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:49.771671 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-943f0772-fca0-4869-b2c2-6ec65bbacf75 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:49.772204 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-943f0772-fca0-4869-b2c2-6ec65bbacf75 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:49.772968 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-943f0772-fca0-4869-b2c2-6ec65bbacf75 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:49.773484 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-943f0772-fca0-4869-b2c2-6ec65bbacf75 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:49.773959 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-943f0772-fca0-4869-b2c2-6ec65bbacf75 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:49.774793 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f2a7d76e-f2f6-4257-818b-b161a7195bde tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:49.775278 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f2a7d76e-f2f6-4257-818b-b161a7195bde tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:49.775696 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f2a7d76e-f2f6-4257-818b-b161a7195bde tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:49.776474 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f2a7d76e-f2f6-4257-818b-b161a7195bde tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:49.776932 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f2a7d76e-f2f6-4257-818b-b161a7195bde tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:49.777351 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f2a7d76e-f2f6-4257-818b-b161a7195bde tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:49.972230 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-943f0772-fca0-4869-b2c2-6ec65bbacf75 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:49.985587 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-943f0772-fca0-4869-b2c2-6ec65bbacf75 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6" status: 200 len: 1516 microversion: 2.1 time: 0.226714 Oct 30 01:00:49.986388 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-f2a7d76e-f2f6-4257-818b-b161a7195bde tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:49.997197 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-f2a7d76e-f2f6-4257-818b-b161a7195bde tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1724 microversion: 2.1 time: 0.233138 Oct 30 01:00:49.997790 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 292/589] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:49 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1724 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 01:00:50.001087 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 298/590] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:49 2022] GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6 => generated 1516 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:50.023226 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-4d2eca65-ae30-414f-accc-a5a4008c19d8 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:50.023953 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-4d2eca65-ae30-414f-accc-a5a4008c19d8 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: db9ac414-31a5-4570-aa3f-37db3dac2049] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:50.035599 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4d2eca65-ae30-414f-accc-a5a4008c19d8 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:50.036136 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4d2eca65-ae30-414f-accc-a5a4008c19d8 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:50.036733 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4d2eca65-ae30-414f-accc-a5a4008c19d8 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:50.038184 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4d2eca65-ae30-414f-accc-a5a4008c19d8 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:50.038915 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4d2eca65-ae30-414f-accc-a5a4008c19d8 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:50.039334 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4d2eca65-ae30-414f-accc-a5a4008c19d8 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:50.238355 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-4d2eca65-ae30-414f-accc-a5a4008c19d8 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:50.246410 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-4d2eca65-ae30-414f-accc-a5a4008c19d8 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049" status: 200 len: 1319 microversion: 2.1 time: 0.229788 Oct 30 01:00:50.247871 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 293/591] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:50 2022] GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049 => generated 1319 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:50.771389 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-8f9bc3f6-1273-412a-9a27-ee4260e0723f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:50.772040 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-8f9bc3f6-1273-412a-9a27-ee4260e0723f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:50.779199 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8f9bc3f6-1273-412a-9a27-ee4260e0723f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:50.779675 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8f9bc3f6-1273-412a-9a27-ee4260e0723f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:50.780138 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8f9bc3f6-1273-412a-9a27-ee4260e0723f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:50.780859 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8f9bc3f6-1273-412a-9a27-ee4260e0723f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:50.781337 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8f9bc3f6-1273-412a-9a27-ee4260e0723f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:50.781757 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8f9bc3f6-1273-412a-9a27-ee4260e0723f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:50.933929 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-8f9bc3f6-1273-412a-9a27-ee4260e0723f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:50.942788 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-8f9bc3f6-1273-412a-9a27-ee4260e0723f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:50.944253 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-8f9bc3f6-1273-412a-9a27-ee4260e0723f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:50.945447 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-8f9bc3f6-1273-412a-9a27-ee4260e0723f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1739 microversion: 2.20 time: 0.178190 Oct 30 01:00:50.946089 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 299/592] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:00:50 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1739 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:00:51.021794 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-6c69c8a7-a560-4634-ae88-72f579b2e908 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:51.022934 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-6c69c8a7-a560-4634-ae88-72f579b2e908 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:51.029584 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6c69c8a7-a560-4634-ae88-72f579b2e908 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:51.030039 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6c69c8a7-a560-4634-ae88-72f579b2e908 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:51.030679 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6c69c8a7-a560-4634-ae88-72f579b2e908 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:51.031392 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6c69c8a7-a560-4634-ae88-72f579b2e908 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:51.031859 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6c69c8a7-a560-4634-ae88-72f579b2e908 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:51.032267 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6c69c8a7-a560-4634-ae88-72f579b2e908 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:51.174883 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-6c69c8a7-a560-4634-ae88-72f579b2e908 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:51.183273 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-4ca6d9c1-e078-4a71-a7fc-c1600b6065d8 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Action: 'create', calling method: .version_select at 0x7f0eabd30820>, body: {"keypair": {"name": "tempest-keypair-457360226", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBEZU34St8IOdGxod/eGNP6xj859bdjlq712s87IDB6ghBtObIEQcLkiqSFBx+wpqow1SpHb06lEZ1St8lDuLHJMg7k/87an6w0b05cbghqDAEdrmNarp+ynLPnSqnUh6jw=="}} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:00:51.185100 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-6c69c8a7-a560-4634-ae88-72f579b2e908 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1724 microversion: 2.1 time: 0.168060 Oct 30 01:00:51.186073 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 294/593] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:51 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1724 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:51.213502 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.quota [None req-4ca6d9c1-e078-4a71-a7fc-c1600b6065d8 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Getting quotas for project 6b68b886c19c45d585409651f33b8626. Resources: {'key_pairs'} {{(pid=118961) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 30 01:00:51.226281 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.quota [None req-4ca6d9c1-e078-4a71-a7fc-c1600b6065d8 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Getting quotas for user ecaf8ec105c04b919256f852da859eba and project 6b68b886c19c45d585409651f33b8626. Resources: {'key_pairs'} {{(pid=118961) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 30 01:00:51.256995 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-4ca6d9c1-e078-4a71-a7fc-c1600b6065d8 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.077482 Oct 30 01:00:51.257681 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 300/594] 10.222.0.16 () {60 vars in 1159 bytes} [Sun Oct 30 01:00:51 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 30 01:00:51.272727 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-53d8ba28-d4e2-4d17-9e93-0ea48fc96ce4 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:51.273251 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-53d8ba28-d4e2-4d17-9e93-0ea48fc96ce4 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: db9ac414-31a5-4570-aa3f-37db3dac2049] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:51.279755 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-53d8ba28-d4e2-4d17-9e93-0ea48fc96ce4 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:51.280056 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-53d8ba28-d4e2-4d17-9e93-0ea48fc96ce4 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:51.280637 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-53d8ba28-d4e2-4d17-9e93-0ea48fc96ce4 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:51.281415 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-53d8ba28-d4e2-4d17-9e93-0ea48fc96ce4 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:51.281905 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-53d8ba28-d4e2-4d17-9e93-0ea48fc96ce4 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:51.282311 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-53d8ba28-d4e2-4d17-9e93-0ea48fc96ce4 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:51.381949 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-7000789f-0902-4942-ba57-4ab63c97bce1 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Action: 'create', calling method: .version_select at 0x7f0eabc28a60>, body: {"keypair": {"name": "tempest-keypair-1855286619", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFREfy6qs5wc1ygjMh2FGRpuobGztc13ZZVAs/82w3Djyfykr3tMX4D+bJax+g3/udqOW80c8R+oN8ZxD6yCxGD4/ZDdy0MVztlT6cgqohVUo7Q7iyJfnjByY1Jc8E3DLg=="}} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:00:51.387580 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=4 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:00:51.401424 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.quota [None req-7000789f-0902-4942-ba57-4ab63c97bce1 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Getting quotas for project f48276ac51dd4bb1912fc670433cf6c4. Resources: {'key_pairs'} {{(pid=118961) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 30 01:00:51.405534 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.quota [None req-7000789f-0902-4942-ba57-4ab63c97bce1 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Getting quotas for user 31b7342ecf07408cab6bb040f361fa1f and project f48276ac51dd4bb1912fc670433cf6c4. Resources: {'key_pairs'} {{(pid=118961) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 30 01:00:51.430215 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-53d8ba28-d4e2-4d17-9e93-0ea48fc96ce4 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:51.430994 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-7000789f-0902-4942-ba57-4ab63c97bce1 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.053106 Oct 30 01:00:51.431608 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 301/595] 10.222.0.16 () {60 vars in 1159 bytes} [Sun Oct 30 01:00:51 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:00:51.440233 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-53d8ba28-d4e2-4d17-9e93-0ea48fc96ce4 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049" status: 200 len: 1319 microversion: 2.1 time: 0.171339 Oct 30 01:00:51.441040 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 295/596] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:51 2022] GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049 => generated 1319 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:51.974826 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-b60eb4e8-3904-493c-a34a-97a76687871d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:51.977073 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-b60eb4e8-3904-493c-a34a-97a76687871d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:51.986509 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b60eb4e8-3904-493c-a34a-97a76687871d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:51.987196 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b60eb4e8-3904-493c-a34a-97a76687871d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:51.987734 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b60eb4e8-3904-493c-a34a-97a76687871d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:51.988885 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b60eb4e8-3904-493c-a34a-97a76687871d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:51.989341 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b60eb4e8-3904-493c-a34a-97a76687871d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:51.989766 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b60eb4e8-3904-493c-a34a-97a76687871d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:52.216747 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-04583ac7-d45d-4d23-82b0-58f7f512aa97 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:52.218250 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-04583ac7-d45d-4d23-82b0-58f7f512aa97 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:52.231392 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-04583ac7-d45d-4d23-82b0-58f7f512aa97 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:52.233051 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-04583ac7-d45d-4d23-82b0-58f7f512aa97 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:52.234084 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-04583ac7-d45d-4d23-82b0-58f7f512aa97 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:52.235468 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-04583ac7-d45d-4d23-82b0-58f7f512aa97 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:52.236411 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-04583ac7-d45d-4d23-82b0-58f7f512aa97 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:52.237631 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-04583ac7-d45d-4d23-82b0-58f7f512aa97 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:52.258296 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-b60eb4e8-3904-493c-a34a-97a76687871d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:52.271070 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-b60eb4e8-3904-493c-a34a-97a76687871d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:52.274024 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-b60eb4e8-3904-493c-a34a-97a76687871d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:52.275629 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-b60eb4e8-3904-493c-a34a-97a76687871d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1739 microversion: 2.20 time: 0.306320 Oct 30 01:00:52.296927 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 302/597] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:00:51 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1739 bytes in 320 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:00:52.464685 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-12a27ff3-a219-459f-a7c4-e2013b7d39bc tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:52.465944 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-12a27ff3-a219-459f-a7c4-e2013b7d39bc tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: db9ac414-31a5-4570-aa3f-37db3dac2049] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:52.477131 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-12a27ff3-a219-459f-a7c4-e2013b7d39bc tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:52.477617 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-12a27ff3-a219-459f-a7c4-e2013b7d39bc tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:52.478076 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-12a27ff3-a219-459f-a7c4-e2013b7d39bc tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:52.478786 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-12a27ff3-a219-459f-a7c4-e2013b7d39bc tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:52.479236 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-12a27ff3-a219-459f-a7c4-e2013b7d39bc tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:52.479762 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-12a27ff3-a219-459f-a7c4-e2013b7d39bc tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:52.515770 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-04583ac7-d45d-4d23-82b0-58f7f512aa97 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:52.525713 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-04583ac7-d45d-4d23-82b0-58f7f512aa97 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1724 microversion: 2.1 time: 0.318921 Oct 30 01:00:52.530555 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 296/598] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:52 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1724 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:52.736035 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-12a27ff3-a219-459f-a7c4-e2013b7d39bc tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:52.745917 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-12a27ff3-a219-459f-a7c4-e2013b7d39bc tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049" status: 200 len: 1319 microversion: 2.1 time: 0.285113 Oct 30 01:00:52.747188 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 303/599] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:52 2022] GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049 => generated 1319 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:53.302908 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-913d3bfe-732f-485f-a71f-cf3d15b0c9f2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:53.305248 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-913d3bfe-732f-485f-a71f-cf3d15b0c9f2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:53.318369 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-913d3bfe-732f-485f-a71f-cf3d15b0c9f2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:53.320784 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-913d3bfe-732f-485f-a71f-cf3d15b0c9f2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:53.325591 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-913d3bfe-732f-485f-a71f-cf3d15b0c9f2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:53.326748 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-913d3bfe-732f-485f-a71f-cf3d15b0c9f2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:53.329793 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-913d3bfe-732f-485f-a71f-cf3d15b0c9f2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:53.330724 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-913d3bfe-732f-485f-a71f-cf3d15b0c9f2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:53.370163 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-33ff585b-ee21-4e13-9702-624a6493fd47 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-105817064", "imageRef": "4c16f4f6-84cb-4463-b82c-b7c65f3826f0", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "4400f15d-ace0-4893-b5ff-c43f47dad5a5"}], "security_groups": [{"name": "tempest-securitygroup--1248987986"}], "key_name": "tempest-keypair-457360226", "user_data": "}} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:00:53.628574 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-913d3bfe-732f-485f-a71f-cf3d15b0c9f2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:53.640061 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-913d3bfe-732f-485f-a71f-cf3d15b0c9f2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:53.641480 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-913d3bfe-732f-485f-a71f-cf3d15b0c9f2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:53.642868 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-913d3bfe-732f-485f-a71f-cf3d15b0c9f2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1739 microversion: 2.20 time: 0.344153 Oct 30 01:00:53.643428 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 297/600] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:00:53 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1739 bytes in 345 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:00:53.648570 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-8a2dc0ab-ec03-4423-b80d-b88f022d3348 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:53.649425 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-8a2dc0ab-ec03-4423-b80d-b88f022d3348 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:53.657315 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8a2dc0ab-ec03-4423-b80d-b88f022d3348 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:53.658612 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8a2dc0ab-ec03-4423-b80d-b88f022d3348 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:53.659260 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8a2dc0ab-ec03-4423-b80d-b88f022d3348 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:53.660152 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8a2dc0ab-ec03-4423-b80d-b88f022d3348 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:53.660774 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8a2dc0ab-ec03-4423-b80d-b88f022d3348 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:53.661338 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8a2dc0ab-ec03-4423-b80d-b88f022d3348 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:53.839109 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-8a2dc0ab-ec03-4423-b80d-b88f022d3348 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:53.848024 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-8a2dc0ab-ec03-4423-b80d-b88f022d3348 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1724 microversion: 2.1 time: 0.203348 Oct 30 01:00:53.848839 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 298/601] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:53 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1724 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:53.853894 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-165d1a13-1122-4a72-a583-9e3eefcb1fdd tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:00:53.854812 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-165d1a13-1122-4a72-a583-9e3eefcb1fdd tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: db9ac414-31a5-4570-aa3f-37db3dac2049] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:00:53.860842 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-165d1a13-1122-4a72-a583-9e3eefcb1fdd tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:53.861518 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-165d1a13-1122-4a72-a583-9e3eefcb1fdd tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:53.862090 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-165d1a13-1122-4a72-a583-9e3eefcb1fdd tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:53.863215 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-165d1a13-1122-4a72-a583-9e3eefcb1fdd tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:00:53.864165 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-165d1a13-1122-4a72-a583-9e3eefcb1fdd tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:00:53.864788 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-165d1a13-1122-4a72-a583-9e3eefcb1fdd tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:00:54.006641 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-165d1a13-1122-4a72-a583-9e3eefcb1fdd tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:00:54.017148 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-165d1a13-1122-4a72-a583-9e3eefcb1fdd tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049" status: 200 len: 1509 microversion: 2.1 time: 0.166664 Oct 30 01:00:54.018269 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 299/602] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:00:53 2022] GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049 => generated 1509 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:00:54.047301 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-09549d6b-86e7-46fc-805c-8c04624d4f24 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-115687839", "imageRef": "4c16f4f6-84cb-4463-b82c-b7c65f3826f0", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "cc0a9d8a-a1b9-41b6-86c4-62a6a6fe159a"}], "security_groups": [{"name": "tempest-securitygroup--1286806254"}], "key_name": "tempest-keypair-1855286619", "user_data": "}} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:01:00.219701 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.network.neutron [None req-33ff585b-ee21-4e13-9702-624a6493fd47 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] validate_networks() for [('4400f15d-ace0-4893-b5ff-c43f47dad5a5', None, None, None, None, None)] {{(pid=118961) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 30 01:01:00.298326 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.network.neutron [None req-09549d6b-86e7-46fc-805c-8c04624d4f24 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] validate_networks() for [('cc0a9d8a-a1b9-41b6-86c4-62a6a6fe159a', None, None, None, None, None)] {{(pid=118962) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 30 01:01:01.165005 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-33ff585b-ee21-4e13-9702-624a6493fd47 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Flavor limits 0:0:0 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 30 01:01:01.165636 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-33ff585b-ee21-4e13-9702-624a6493fd47 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Image limits 0:0:0 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 30 01:01:01.166272 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-33ff585b-ee21-4e13-9702-624a6493fd47 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Flavor pref 0:0:0 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 30 01:01:01.166837 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-33ff585b-ee21-4e13-9702-624a6493fd47 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Image pref 0:0:0 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 30 01:01:01.167446 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-33ff585b-ee21-4e13-9702-624a6493fd47 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 30 01:01:01.170027 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.virt.hardware [None req-09549d6b-86e7-46fc-805c-8c04624d4f24 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Flavor limits 0:0:0 {{(pid=118962) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 30 01:01:01.170455 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.virt.hardware [None req-09549d6b-86e7-46fc-805c-8c04624d4f24 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Image limits 0:0:0 {{(pid=118962) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 30 01:01:01.170940 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.virt.hardware [None req-09549d6b-86e7-46fc-805c-8c04624d4f24 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Flavor pref 0:0:0 {{(pid=118962) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 30 01:01:01.171347 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.virt.hardware [None req-09549d6b-86e7-46fc-805c-8c04624d4f24 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Image pref 0:0:0 {{(pid=118962) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 30 01:01:01.171773 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.virt.hardware [None req-09549d6b-86e7-46fc-805c-8c04624d4f24 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118962) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 30 01:01:01.181691 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-09549d6b-86e7-46fc-805c-8c04624d4f24 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:01.182195 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-09549d6b-86e7-46fc-805c-8c04624d4f24 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:01.182617 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-09549d6b-86e7-46fc-805c-8c04624d4f24 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:01.183449 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-09549d6b-86e7-46fc-805c-8c04624d4f24 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:01.183953 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-09549d6b-86e7-46fc-805c-8c04624d4f24 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:01.184781 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-09549d6b-86e7-46fc-805c-8c04624d4f24 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:01.193613 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-33ff585b-ee21-4e13-9702-624a6493fd47 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:01.194395 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-33ff585b-ee21-4e13-9702-624a6493fd47 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:01.194930 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-33ff585b-ee21-4e13-9702-624a6493fd47 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:01.197746 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-33ff585b-ee21-4e13-9702-624a6493fd47 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:01.198541 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-33ff585b-ee21-4e13-9702-624a6493fd47 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:01.199228 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-33ff585b-ee21-4e13-9702-624a6493fd47 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:01.206870 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.quota [None req-09549d6b-86e7-46fc-805c-8c04624d4f24 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Getting quotas for project f48276ac51dd4bb1912fc670433cf6c4. Resources: {'ram', 'instances', 'cores'} {{(pid=118962) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 30 01:01:01.211426 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.quota [None req-09549d6b-86e7-46fc-805c-8c04624d4f24 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Getting quotas for user 31b7342ecf07408cab6bb040f361fa1f and project f48276ac51dd4bb1912fc670433cf6c4. Resources: {'ram', 'instances', 'cores'} {{(pid=118962) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 30 01:01:01.225299 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-09549d6b-86e7-46fc-805c-8c04624d4f24 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Going to run 1 instances... {{(pid=118962) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 30 01:01:01.227625 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-09549d6b-86e7-46fc-805c-8c04624d4f24 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4c16f4f6-84cb-4463-b82c-b7c65f3826f0',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118962) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 30 01:01:01.241981 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.quota [None req-33ff585b-ee21-4e13-9702-624a6493fd47 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Getting quotas for project 6b68b886c19c45d585409651f33b8626. Resources: {'ram', 'instances', 'cores'} {{(pid=118961) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 30 01:01:01.247841 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.quota [None req-33ff585b-ee21-4e13-9702-624a6493fd47 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Getting quotas for user ecaf8ec105c04b919256f852da859eba and project 6b68b886c19c45d585409651f33b8626. Resources: {'ram', 'instances', 'cores'} {{(pid=118961) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 30 01:01:01.262863 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-33ff585b-ee21-4e13-9702-624a6493fd47 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Going to run 1 instances... {{(pid=118961) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 30 01:01:01.266140 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-33ff585b-ee21-4e13-9702-624a6493fd47 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5632075c-7062-434e-b00c-bbf04c5ac04a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4c16f4f6-84cb-4463-b82c-b7c65f3826f0',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118961) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 30 01:01:01.279312 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-09549d6b-86e7-46fc-805c-8c04624d4f24 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 409 microversion: 2.1 time: 7.242233 Oct 30 01:01:01.279999 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 300/603] 10.222.0.16 () {60 vars in 1147 bytes} [Sun Oct 30 01:00:54 2022] POST /compute/v2.1/servers => generated 409 bytes in 7261 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 30 01:01:01.284969 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-e888f0e6-152f-48f0-b618-257d384e954e tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "2b91b9a3-f9a3-43fd-a1ba-3be33aa06d1e", "device": "/dev/sdb"}} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:01:01.287475 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-e888f0e6-152f-48f0-b618-257d384e954e tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: db9ac414-31a5-4570-aa3f-37db3dac2049] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:01.294734 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e888f0e6-152f-48f0-b618-257d384e954e tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:01.295482 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e888f0e6-152f-48f0-b618-257d384e954e tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:01.296119 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e888f0e6-152f-48f0-b618-257d384e954e tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:01.297164 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e888f0e6-152f-48f0-b618-257d384e954e tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:01.297837 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e888f0e6-152f-48f0-b618-257d384e954e tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:01.298472 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e888f0e6-152f-48f0-b618-257d384e954e tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:01.392564 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-33ff585b-ee21-4e13-9702-624a6493fd47 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 8.027392 Oct 30 01:01:01.396181 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 304/604] 10.222.0.16 () {60 vars in 1147 bytes} [Sun Oct 30 01:00:53 2022] POST /compute/v2.1/servers => generated 402 bytes in 8031 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 30 01:01:01.663376 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-4ee04856-2194-43e4-83c8-532cdb3e3fe6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:01.672263 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-4ee04856-2194-43e4-83c8-532cdb3e3fe6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:01.692086 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4ee04856-2194-43e4-83c8-532cdb3e3fe6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:01.697538 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4ee04856-2194-43e4-83c8-532cdb3e3fe6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:01.698292 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4ee04856-2194-43e4-83c8-532cdb3e3fe6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:01.700067 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4ee04856-2194-43e4-83c8-532cdb3e3fe6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:01.703884 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4ee04856-2194-43e4-83c8-532cdb3e3fe6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:01.704623 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4ee04856-2194-43e4-83c8-532cdb3e3fe6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:01.734951 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:01:01.762165 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-e888f0e6-152f-48f0-b618-257d384e954e tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "POST /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.480580 Oct 30 01:01:01.764192 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 301/605] 10.222.0.16 () {60 vars in 1323 bytes} [Sun Oct 30 01:01:01 2022] POST /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049/os-volume_attachments => generated 194 bytes in 483 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:01:02.025951 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-4ee04856-2194-43e4-83c8-532cdb3e3fe6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:02.033075 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-41d08960-2c46-487e-876f-d04eb7c7b5ea tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:02.034312 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-41d08960-2c46-487e-876f-d04eb7c7b5ea tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:02.039604 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-4ee04856-2194-43e4-83c8-532cdb3e3fe6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:02.041735 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-4ee04856-2194-43e4-83c8-532cdb3e3fe6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:02.042802 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-41d08960-2c46-487e-876f-d04eb7c7b5ea tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:02.043337 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-41d08960-2c46-487e-876f-d04eb7c7b5ea tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:02.044456 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-41d08960-2c46-487e-876f-d04eb7c7b5ea tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:02.045444 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-41d08960-2c46-487e-876f-d04eb7c7b5ea tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:02.045817 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-41d08960-2c46-487e-876f-d04eb7c7b5ea tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:02.046353 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-41d08960-2c46-487e-876f-d04eb7c7b5ea tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:02.048526 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-4ee04856-2194-43e4-83c8-532cdb3e3fe6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1739 microversion: 2.20 time: 0.651040 Oct 30 01:01:02.050507 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 305/606] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:01:01 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1739 bytes in 654 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:01:02.061256 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-06d436d0-e619-421d-927c-edf8b9817d00 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:02.061915 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-06d436d0-e619-421d-927c-edf8b9817d00 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:02.077656 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-06d436d0-e619-421d-927c-edf8b9817d00 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:02.078266 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-06d436d0-e619-421d-927c-edf8b9817d00 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:02.078697 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-06d436d0-e619-421d-927c-edf8b9817d00 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:02.079530 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-06d436d0-e619-421d-927c-edf8b9817d00 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:02.080025 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-06d436d0-e619-421d-927c-edf8b9817d00 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:02.080471 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-06d436d0-e619-421d-927c-edf8b9817d00 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:02.171339 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-06d436d0-e619-421d-927c-edf8b9817d00 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:02.194056 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-06d436d0-e619-421d-927c-edf8b9817d00 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1249 microversion: 2.1 time: 0.133431 Oct 30 01:01:02.194739 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 306/607] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:02 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1249 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:02.205308 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-9de1570e-8e04-4f42-a763-c404374e498a tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:02.206031 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-9de1570e-8e04-4f42-a763-c404374e498a tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5632075c-7062-434e-b00c-bbf04c5ac04a] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:02.215904 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9de1570e-8e04-4f42-a763-c404374e498a tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:02.216836 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9de1570e-8e04-4f42-a763-c404374e498a tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:02.217687 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9de1570e-8e04-4f42-a763-c404374e498a tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:02.218761 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9de1570e-8e04-4f42-a763-c404374e498a tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:02.219432 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9de1570e-8e04-4f42-a763-c404374e498a tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:02.220088 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9de1570e-8e04-4f42-a763-c404374e498a tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:02.333436 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-9de1570e-8e04-4f42-a763-c404374e498a tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:02.346078 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-9de1570e-8e04-4f42-a763-c404374e498a tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a" status: 200 len: 1244 microversion: 2.1 time: 0.144438 Oct 30 01:01:02.348127 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 307/608] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:02 2022] GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a => generated 1244 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:02.440684 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-41d08960-2c46-487e-876f-d04eb7c7b5ea tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:02.450178 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-41d08960-2c46-487e-876f-d04eb7c7b5ea tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1701 microversion: 2.1 time: 0.683767 Oct 30 01:01:02.469755 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 302/609] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:01 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1701 bytes in 704 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:02.476696 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-15acd65a-7af3-49dd-b7d5-caff3765bc4b tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:01:02.483446 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-15acd65a-7af3-49dd-b7d5-caff3765bc4b tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:02.484069 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-15acd65a-7af3-49dd-b7d5-caff3765bc4b tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:02.484792 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-15acd65a-7af3-49dd-b7d5-caff3765bc4b tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:02.511658 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-15acd65a-7af3-49dd-b7d5-caff3765bc4b tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Going to try to start instance {{(pid=118961) start /opt/stack/nova/nova/compute/api.py:2785}} Oct 30 01:01:02.567140 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-15acd65a-7af3-49dd-b7d5-caff3765bc4b tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "POST /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14/action" status: 202 len: 0 microversion: 2.1 time: 0.096601 Oct 30 01:01:02.571507 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 308/610] 10.222.0.16 () {60 vars in 1278 bytes} [Sun Oct 30 01:01:02 2022] POST /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14/action => generated 0 bytes in 99 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 30 01:01:02.578383 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-2dec418b-15f8-4b6d-8ddb-a3fbc3ebf03e tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:02.580014 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-2dec418b-15f8-4b6d-8ddb-a3fbc3ebf03e tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:02.589473 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2dec418b-15f8-4b6d-8ddb-a3fbc3ebf03e tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:02.590089 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2dec418b-15f8-4b6d-8ddb-a3fbc3ebf03e tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:02.590944 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2dec418b-15f8-4b6d-8ddb-a3fbc3ebf03e tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:02.591945 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2dec418b-15f8-4b6d-8ddb-a3fbc3ebf03e tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:02.592605 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2dec418b-15f8-4b6d-8ddb-a3fbc3ebf03e tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:02.593433 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2dec418b-15f8-4b6d-8ddb-a3fbc3ebf03e tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:02.828597 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-2dec418b-15f8-4b6d-8ddb-a3fbc3ebf03e tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:02.839730 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-2dec418b-15f8-4b6d-8ddb-a3fbc3ebf03e tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1710 microversion: 2.1 time: 0.265948 Oct 30 01:01:02.847758 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 303/611] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:02 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1710 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:03.075107 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-3292f595-a41b-43f6-951a-45bf1b33aaab tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:03.075936 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-3292f595-a41b-43f6-951a-45bf1b33aaab tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:03.084349 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3292f595-a41b-43f6-951a-45bf1b33aaab tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:03.085180 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3292f595-a41b-43f6-951a-45bf1b33aaab tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:03.085860 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3292f595-a41b-43f6-951a-45bf1b33aaab tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:03.086939 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3292f595-a41b-43f6-951a-45bf1b33aaab tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:03.087616 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3292f595-a41b-43f6-951a-45bf1b33aaab tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:03.088254 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3292f595-a41b-43f6-951a-45bf1b33aaab tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:03.222486 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-211a9bc0-f839-4a85-addd-594723e0675e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:03.224217 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-211a9bc0-f839-4a85-addd-594723e0675e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:03.231250 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-211a9bc0-f839-4a85-addd-594723e0675e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:03.232253 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-211a9bc0-f839-4a85-addd-594723e0675e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:03.232881 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-211a9bc0-f839-4a85-addd-594723e0675e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:03.233779 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-211a9bc0-f839-4a85-addd-594723e0675e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:03.234406 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-211a9bc0-f839-4a85-addd-594723e0675e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:03.235008 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-211a9bc0-f839-4a85-addd-594723e0675e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:03.301219 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-3292f595-a41b-43f6-951a-45bf1b33aaab tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:03.315006 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-3292f595-a41b-43f6-951a-45bf1b33aaab tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:03.324109 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-3292f595-a41b-43f6-951a-45bf1b33aaab tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:03.328437 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-3292f595-a41b-43f6-951a-45bf1b33aaab tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1739 microversion: 2.20 time: 0.258247 Oct 30 01:01:03.344668 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 309/612] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:01:03 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1739 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:01:03.377784 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-25a568ab-2bda-4158-897d-5a0b000a413a tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:03.378502 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-25a568ab-2bda-4158-897d-5a0b000a413a tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5632075c-7062-434e-b00c-bbf04c5ac04a] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:03.386239 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-211a9bc0-f839-4a85-addd-594723e0675e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:03.398452 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-211a9bc0-f839-4a85-addd-594723e0675e tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1297 microversion: 2.1 time: 0.181560 Oct 30 01:01:03.399350 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 304/613] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:03 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1297 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:03.401954 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-25a568ab-2bda-4158-897d-5a0b000a413a tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:03.403490 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-25a568ab-2bda-4158-897d-5a0b000a413a tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:03.405518 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-25a568ab-2bda-4158-897d-5a0b000a413a tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:03.406264 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-25a568ab-2bda-4158-897d-5a0b000a413a tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:03.407809 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-25a568ab-2bda-4158-897d-5a0b000a413a tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:03.410326 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-25a568ab-2bda-4158-897d-5a0b000a413a tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:03.514507 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-25a568ab-2bda-4158-897d-5a0b000a413a tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:03.523332 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-25a568ab-2bda-4158-897d-5a0b000a413a tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a" status: 200 len: 1292 microversion: 2.1 time: 0.150144 Oct 30 01:01:03.526950 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 310/614] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:03 2022] GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a => generated 1292 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 01:01:03.863275 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-d0d0ca2a-88d1-4ce6-bcda-9482cd2cebc9 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:03.864741 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-d0d0ca2a-88d1-4ce6-bcda-9482cd2cebc9 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:03.872614 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d0d0ca2a-88d1-4ce6-bcda-9482cd2cebc9 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:03.874036 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d0d0ca2a-88d1-4ce6-bcda-9482cd2cebc9 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:03.874891 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d0d0ca2a-88d1-4ce6-bcda-9482cd2cebc9 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:03.876186 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d0d0ca2a-88d1-4ce6-bcda-9482cd2cebc9 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:03.876647 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d0d0ca2a-88d1-4ce6-bcda-9482cd2cebc9 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:03.877067 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d0d0ca2a-88d1-4ce6-bcda-9482cd2cebc9 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:04.016176 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-d0d0ca2a-88d1-4ce6-bcda-9482cd2cebc9 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:04.024387 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-d0d0ca2a-88d1-4ce6-bcda-9482cd2cebc9 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1710 microversion: 2.1 time: 0.165620 Oct 30 01:01:04.024952 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 305/615] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:03 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1710 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:04.359065 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-71433d85-399a-4ffa-9ef8-6825ba687bbd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:04.359690 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-71433d85-399a-4ffa-9ef8-6825ba687bbd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:04.367184 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-71433d85-399a-4ffa-9ef8-6825ba687bbd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:04.368040 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-71433d85-399a-4ffa-9ef8-6825ba687bbd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:04.368642 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-71433d85-399a-4ffa-9ef8-6825ba687bbd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:04.369698 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-71433d85-399a-4ffa-9ef8-6825ba687bbd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:04.370203 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-71433d85-399a-4ffa-9ef8-6825ba687bbd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:04.370672 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-71433d85-399a-4ffa-9ef8-6825ba687bbd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:04.421841 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-9e45fd74-838b-4de1-82a0-ea29f0b6f48c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:04.422811 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-9e45fd74-838b-4de1-82a0-ea29f0b6f48c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:04.428944 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9e45fd74-838b-4de1-82a0-ea29f0b6f48c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:04.429429 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9e45fd74-838b-4de1-82a0-ea29f0b6f48c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:04.429859 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9e45fd74-838b-4de1-82a0-ea29f0b6f48c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:04.430550 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9e45fd74-838b-4de1-82a0-ea29f0b6f48c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:04.431274 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9e45fd74-838b-4de1-82a0-ea29f0b6f48c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:04.431667 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9e45fd74-838b-4de1-82a0-ea29f0b6f48c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:04.506336 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-9e45fd74-838b-4de1-82a0-ea29f0b6f48c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:04.514469 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-9e45fd74-838b-4de1-82a0-ea29f0b6f48c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1303 microversion: 2.1 time: 0.096508 Oct 30 01:01:04.515061 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 306/616] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:04 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1303 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:04.528463 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-71433d85-399a-4ffa-9ef8-6825ba687bbd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:04.537044 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-71433d85-399a-4ffa-9ef8-6825ba687bbd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:04.538480 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-71433d85-399a-4ffa-9ef8-6825ba687bbd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:04.539659 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-71433d85-399a-4ffa-9ef8-6825ba687bbd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1739 microversion: 2.20 time: 0.184746 Oct 30 01:01:04.540382 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 311/617] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:01:04 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1739 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:01:04.552087 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-8c7bab18-344d-45ae-a7f4-0b5f0ee1368d tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:04.552985 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-8c7bab18-344d-45ae-a7f4-0b5f0ee1368d tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5632075c-7062-434e-b00c-bbf04c5ac04a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:04.559170 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8c7bab18-344d-45ae-a7f4-0b5f0ee1368d tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:04.559806 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8c7bab18-344d-45ae-a7f4-0b5f0ee1368d tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:04.560255 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8c7bab18-344d-45ae-a7f4-0b5f0ee1368d tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:04.560958 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8c7bab18-344d-45ae-a7f4-0b5f0ee1368d tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:04.561399 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8c7bab18-344d-45ae-a7f4-0b5f0ee1368d tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:04.561803 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8c7bab18-344d-45ae-a7f4-0b5f0ee1368d tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:04.632855 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-8c7bab18-344d-45ae-a7f4-0b5f0ee1368d tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:04.641029 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-8c7bab18-344d-45ae-a7f4-0b5f0ee1368d tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a" status: 200 len: 1292 microversion: 2.1 time: 0.093607 Oct 30 01:01:04.641603 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 307/618] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:04 2022] GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a => generated 1292 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 01:01:05.051747 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-4c330f63-9f5c-4fbb-85df-cc0460d7c6d9 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:05.052782 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-4c330f63-9f5c-4fbb-85df-cc0460d7c6d9 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:05.061340 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4c330f63-9f5c-4fbb-85df-cc0460d7c6d9 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:05.061806 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4c330f63-9f5c-4fbb-85df-cc0460d7c6d9 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:05.062274 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4c330f63-9f5c-4fbb-85df-cc0460d7c6d9 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:05.062981 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4c330f63-9f5c-4fbb-85df-cc0460d7c6d9 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:05.063440 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4c330f63-9f5c-4fbb-85df-cc0460d7c6d9 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:05.063982 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4c330f63-9f5c-4fbb-85df-cc0460d7c6d9 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:05.199263 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-4c330f63-9f5c-4fbb-85df-cc0460d7c6d9 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:05.203080 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:01:05.208525 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-4c330f63-9f5c-4fbb-85df-cc0460d7c6d9 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1710 microversion: 2.1 time: 0.163352 Oct 30 01:01:05.209112 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 312/619] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:05 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1710 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:05.539238 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-eb6fb21f-f3a0-4dde-8ab1-72095af01661 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:05.539842 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-eb6fb21f-f3a0-4dde-8ab1-72095af01661 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:05.546465 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-eb6fb21f-f3a0-4dde-8ab1-72095af01661 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:05.546952 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-eb6fb21f-f3a0-4dde-8ab1-72095af01661 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:05.547389 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-eb6fb21f-f3a0-4dde-8ab1-72095af01661 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:05.548113 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-eb6fb21f-f3a0-4dde-8ab1-72095af01661 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:05.548564 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-eb6fb21f-f3a0-4dde-8ab1-72095af01661 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:05.548981 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-eb6fb21f-f3a0-4dde-8ab1-72095af01661 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:05.566331 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-eb8746aa-b87a-452b-9096-2c0ae54ce84c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:05.567283 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-eb8746aa-b87a-452b-9096-2c0ae54ce84c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:05.575651 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-eb8746aa-b87a-452b-9096-2c0ae54ce84c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:05.576243 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-eb8746aa-b87a-452b-9096-2c0ae54ce84c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:05.577123 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-eb8746aa-b87a-452b-9096-2c0ae54ce84c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:05.578195 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-eb8746aa-b87a-452b-9096-2c0ae54ce84c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:05.579090 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-eb8746aa-b87a-452b-9096-2c0ae54ce84c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:05.579520 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-eb8746aa-b87a-452b-9096-2c0ae54ce84c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:05.754437 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-eb6fb21f-f3a0-4dde-8ab1-72095af01661 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:05.764774 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-eb6fb21f-f3a0-4dde-8ab1-72095af01661 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1371 microversion: 2.1 time: 0.230299 Oct 30 01:01:05.780243 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 308/620] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:05 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1371 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:05.782606 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-eb8746aa-b87a-452b-9096-2c0ae54ce84c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:05.787618 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-69bd5c9b-010c-44d3-b69e-8ee41b560a6c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:05.788632 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-69bd5c9b-010c-44d3-b69e-8ee41b560a6c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5632075c-7062-434e-b00c-bbf04c5ac04a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:05.796474 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-69bd5c9b-010c-44d3-b69e-8ee41b560a6c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:05.796971 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-69bd5c9b-010c-44d3-b69e-8ee41b560a6c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:05.797404 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-69bd5c9b-010c-44d3-b69e-8ee41b560a6c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:05.798113 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-69bd5c9b-010c-44d3-b69e-8ee41b560a6c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:05.798566 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-69bd5c9b-010c-44d3-b69e-8ee41b560a6c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:05.799101 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-69bd5c9b-010c-44d3-b69e-8ee41b560a6c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:05.799202 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-eb8746aa-b87a-452b-9096-2c0ae54ce84c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:05.800778 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-eb8746aa-b87a-452b-9096-2c0ae54ce84c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:05.802056 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-eb8746aa-b87a-452b-9096-2c0ae54ce84c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1721 microversion: 2.20 time: 0.239889 Oct 30 01:01:05.803326 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 313/621] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:01:05 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1721 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 30 01:01:05.866026 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-69bd5c9b-010c-44d3-b69e-8ee41b560a6c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:05.874493 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-69bd5c9b-010c-44d3-b69e-8ee41b560a6c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a" status: 200 len: 1300 microversion: 2.1 time: 0.093048 Oct 30 01:01:05.875179 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 309/622] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:05 2022] GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a => generated 1300 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:06.233324 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-4ec8a72d-3430-4785-a4e6-1a31b1bc9e64 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:06.233938 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-4ec8a72d-3430-4785-a4e6-1a31b1bc9e64 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:06.240032 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4ec8a72d-3430-4785-a4e6-1a31b1bc9e64 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:06.240515 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4ec8a72d-3430-4785-a4e6-1a31b1bc9e64 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:06.240974 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4ec8a72d-3430-4785-a4e6-1a31b1bc9e64 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:06.241690 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4ec8a72d-3430-4785-a4e6-1a31b1bc9e64 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:06.242132 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4ec8a72d-3430-4785-a4e6-1a31b1bc9e64 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:06.242546 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4ec8a72d-3430-4785-a4e6-1a31b1bc9e64 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:06.373310 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-4ec8a72d-3430-4785-a4e6-1a31b1bc9e64 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:06.382038 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-4ec8a72d-3430-4785-a4e6-1a31b1bc9e64 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1710 microversion: 2.1 time: 0.152902 Oct 30 01:01:06.382644 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 314/623] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:06 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1710 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:06.436426 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [req-9275302a-4d2e-47f2-9ea9-6fc44a74d87c req-3edc2ab5-37b9-4f82-9975-88303a2cfc6a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "de41579d-0173-4dba-962d-a28c85c505de", "tag": "c4782435-5ac9-430e-95c9-c061a2747b76"}]} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:01:06.446313 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-9275302a-4d2e-47f2-9ea9-6fc44a74d87c req-3edc2ab5-37b9-4f82-9975-88303a2cfc6a service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:06.446854 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-9275302a-4d2e-47f2-9ea9-6fc44a74d87c req-3edc2ab5-37b9-4f82-9975-88303a2cfc6a service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:06.447334 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-9275302a-4d2e-47f2-9ea9-6fc44a74d87c req-3edc2ab5-37b9-4f82-9975-88303a2cfc6a service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:06.458555 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.compute.server_external_events [req-9275302a-4d2e-47f2-9ea9-6fc44a74d87c req-3edc2ab5-37b9-4f82-9975-88303a2cfc6a service nova] Creating event network-changed:c4782435-5ac9-430e-95c9-c061a2747b76 for instance de41579d-0173-4dba-962d-a28c85c505de on hv-cinder-maste Oct 30 01:01:06.463282 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [req-9275302a-4d2e-47f2-9ea9-6fc44a74d87c req-3edc2ab5-37b9-4f82-9975-88303a2cfc6a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030628 Oct 30 01:01:06.463900 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 310/624] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:01:06 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:01:06.791361 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-e57aac84-dc9c-4d2b-8fa5-629484b9a381 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:06.791980 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-e57aac84-dc9c-4d2b-8fa5-629484b9a381 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:06.799110 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e57aac84-dc9c-4d2b-8fa5-629484b9a381 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:06.799596 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e57aac84-dc9c-4d2b-8fa5-629484b9a381 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:06.800075 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e57aac84-dc9c-4d2b-8fa5-629484b9a381 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:06.800769 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e57aac84-dc9c-4d2b-8fa5-629484b9a381 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:06.801222 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e57aac84-dc9c-4d2b-8fa5-629484b9a381 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:06.801642 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e57aac84-dc9c-4d2b-8fa5-629484b9a381 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:06.830854 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-2e31b777-b352-4d85-876a-2df9eb45ea8e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:06.831633 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-2e31b777-b352-4d85-876a-2df9eb45ea8e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:06.838571 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2e31b777-b352-4d85-876a-2df9eb45ea8e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:06.839037 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2e31b777-b352-4d85-876a-2df9eb45ea8e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:06.839457 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2e31b777-b352-4d85-876a-2df9eb45ea8e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:06.840169 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2e31b777-b352-4d85-876a-2df9eb45ea8e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:06.840613 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2e31b777-b352-4d85-876a-2df9eb45ea8e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:06.841025 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2e31b777-b352-4d85-876a-2df9eb45ea8e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:06.991884 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-e57aac84-dc9c-4d2b-8fa5-629484b9a381 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:07.003622 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-e57aac84-dc9c-4d2b-8fa5-629484b9a381 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1371 microversion: 2.1 time: 0.215760 Oct 30 01:01:07.005549 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 315/625] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:06 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1371 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:07.010419 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-08ad260b-3db3-4ca2-8f4b-2f5cfccfc2d5 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:07.011351 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-08ad260b-3db3-4ca2-8f4b-2f5cfccfc2d5 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5632075c-7062-434e-b00c-bbf04c5ac04a] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:07.018955 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-08ad260b-3db3-4ca2-8f4b-2f5cfccfc2d5 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:07.019661 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-08ad260b-3db3-4ca2-8f4b-2f5cfccfc2d5 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:07.020286 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-08ad260b-3db3-4ca2-8f4b-2f5cfccfc2d5 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:07.021284 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-08ad260b-3db3-4ca2-8f4b-2f5cfccfc2d5 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:07.021942 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-08ad260b-3db3-4ca2-8f4b-2f5cfccfc2d5 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:07.022415 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-08ad260b-3db3-4ca2-8f4b-2f5cfccfc2d5 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:07.030035 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-2e31b777-b352-4d85-876a-2df9eb45ea8e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:07.039869 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-2e31b777-b352-4d85-876a-2df9eb45ea8e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:07.041284 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-2e31b777-b352-4d85-876a-2df9eb45ea8e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:07.042564 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-2e31b777-b352-4d85-876a-2df9eb45ea8e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1721 microversion: 2.20 time: 0.216484 Oct 30 01:01:07.043054 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 311/626] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:01:06 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1721 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:01:07.107242 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-08ad260b-3db3-4ca2-8f4b-2f5cfccfc2d5 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:07.115612 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-08ad260b-3db3-4ca2-8f4b-2f5cfccfc2d5 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a" status: 200 len: 1298 microversion: 2.1 time: 0.108789 Oct 30 01:01:07.116349 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 316/627] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:07 2022] GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a => generated 1298 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:07.406068 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-e1f94ced-5d37-4ff1-ac2e-88b1a4290dfd tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:07.406678 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-e1f94ced-5d37-4ff1-ac2e-88b1a4290dfd tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:07.412923 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e1f94ced-5d37-4ff1-ac2e-88b1a4290dfd tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:07.413435 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e1f94ced-5d37-4ff1-ac2e-88b1a4290dfd tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:07.413865 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e1f94ced-5d37-4ff1-ac2e-88b1a4290dfd tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:07.414553 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e1f94ced-5d37-4ff1-ac2e-88b1a4290dfd tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:07.414972 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e1f94ced-5d37-4ff1-ac2e-88b1a4290dfd tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:07.415419 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e1f94ced-5d37-4ff1-ac2e-88b1a4290dfd tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:07.565532 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-e1f94ced-5d37-4ff1-ac2e-88b1a4290dfd tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:07.576725 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-e1f94ced-5d37-4ff1-ac2e-88b1a4290dfd tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1714 microversion: 2.1 time: 0.174653 Oct 30 01:01:07.587454 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 312/628] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:07 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1714 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:07.636626 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [req-d8affc1e-f315-4728-99c9-f3c04f28d183 req-f48dd4d9-ab64-423e-9a45-c93b3fb24f59 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9491a8f4-2b76-4a4e-a6bd-6720114e482f", "name": "network-vif-unplugged", "status": "completed", "tag": "5367850e-73d0-4814-af96-bd430c94c272"}]} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:01:07.646444 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-d8affc1e-f315-4728-99c9-f3c04f28d183 req-f48dd4d9-ab64-423e-9a45-c93b3fb24f59 service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:07.646917 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-d8affc1e-f315-4728-99c9-f3c04f28d183 req-f48dd4d9-ab64-423e-9a45-c93b3fb24f59 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:07.647459 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-d8affc1e-f315-4728-99c9-f3c04f28d183 req-f48dd4d9-ab64-423e-9a45-c93b3fb24f59 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:07.659796 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.compute.server_external_events [req-d8affc1e-f315-4728-99c9-f3c04f28d183 req-f48dd4d9-ab64-423e-9a45-c93b3fb24f59 service nova] Creating event network-vif-unplugged:5367850e-73d0-4814-af96-bd430c94c272 for instance 9491a8f4-2b76-4a4e-a6bd-6720114e482f on hv-cinder-maste Oct 30 01:01:07.664692 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [req-d8affc1e-f315-4728-99c9-f3c04f28d183 req-f48dd4d9-ab64-423e-9a45-c93b3fb24f59 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.032049 Oct 30 01:01:07.665382 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 317/629] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:01:07 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:01:08.027658 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-dec71505-d3c4-4142-aa35-e3bc570ea74c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:08.028603 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-dec71505-d3c4-4142-aa35-e3bc570ea74c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:08.035477 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dec71505-d3c4-4142-aa35-e3bc570ea74c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:08.036003 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dec71505-d3c4-4142-aa35-e3bc570ea74c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:08.036429 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dec71505-d3c4-4142-aa35-e3bc570ea74c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:08.037263 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dec71505-d3c4-4142-aa35-e3bc570ea74c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:08.037721 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dec71505-d3c4-4142-aa35-e3bc570ea74c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:08.038136 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dec71505-d3c4-4142-aa35-e3bc570ea74c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:08.069506 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-17e4a93a-ec2f-4fb7-ae05-6a005eb4e4d2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:08.070258 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-17e4a93a-ec2f-4fb7-ae05-6a005eb4e4d2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:08.076548 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-17e4a93a-ec2f-4fb7-ae05-6a005eb4e4d2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:08.077033 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-17e4a93a-ec2f-4fb7-ae05-6a005eb4e4d2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:08.078034 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-17e4a93a-ec2f-4fb7-ae05-6a005eb4e4d2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:08.078734 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-17e4a93a-ec2f-4fb7-ae05-6a005eb4e4d2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:08.079181 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-17e4a93a-ec2f-4fb7-ae05-6a005eb4e4d2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:08.079801 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-17e4a93a-ec2f-4fb7-ae05-6a005eb4e4d2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:08.234441 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-dec71505-d3c4-4142-aa35-e3bc570ea74c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:08.242827 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-dec71505-d3c4-4142-aa35-e3bc570ea74c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1371 microversion: 2.1 time: 0.218962 Oct 30 01:01:08.243548 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 313/630] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:08 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1371 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:08.249341 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-18415464-419b-49c9-9470-303219fc4093 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:08.250231 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-18415464-419b-49c9-9470-303219fc4093 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5632075c-7062-434e-b00c-bbf04c5ac04a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:08.256699 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-18415464-419b-49c9-9470-303219fc4093 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:08.257411 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-18415464-419b-49c9-9470-303219fc4093 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:08.258798 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-18415464-419b-49c9-9470-303219fc4093 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:08.259703 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-18415464-419b-49c9-9470-303219fc4093 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:08.260392 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-18415464-419b-49c9-9470-303219fc4093 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:08.261004 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-18415464-419b-49c9-9470-303219fc4093 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:08.278311 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-17e4a93a-ec2f-4fb7-ae05-6a005eb4e4d2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:08.300671 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-17e4a93a-ec2f-4fb7-ae05-6a005eb4e4d2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:08.306912 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-17e4a93a-ec2f-4fb7-ae05-6a005eb4e4d2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:08.315775 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-17e4a93a-ec2f-4fb7-ae05-6a005eb4e4d2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1721 microversion: 2.20 time: 0.248065 Oct 30 01:01:08.332239 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 318/631] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:01:08 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1721 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:01:08.419808 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-18415464-419b-49c9-9470-303219fc4093 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:08.427752 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-18415464-419b-49c9-9470-303219fc4093 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a" status: 200 len: 1366 microversion: 2.1 time: 0.182882 Oct 30 01:01:08.428338 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 314/632] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:08 2022] GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a => generated 1366 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:09.041584 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [req-3cd48ec2-4a63-4b50-9fd6-8087b968f8ce req-562fed46-7e29-4d78-bb21-5a7c5a00ea84 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5632075c-7062-434e-b00c-bbf04c5ac04a", "tag": "c9fe8633-400b-4adf-99ff-00c04190ea67"}]} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:01:09.051572 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-3cd48ec2-4a63-4b50-9fd6-8087b968f8ce req-562fed46-7e29-4d78-bb21-5a7c5a00ea84 service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:09.052081 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-3cd48ec2-4a63-4b50-9fd6-8087b968f8ce req-562fed46-7e29-4d78-bb21-5a7c5a00ea84 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:09.052480 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-3cd48ec2-4a63-4b50-9fd6-8087b968f8ce req-562fed46-7e29-4d78-bb21-5a7c5a00ea84 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:09.063644 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.compute.server_external_events [req-3cd48ec2-4a63-4b50-9fd6-8087b968f8ce req-562fed46-7e29-4d78-bb21-5a7c5a00ea84 service nova] Creating event network-changed:c9fe8633-400b-4adf-99ff-00c04190ea67 for instance 5632075c-7062-434e-b00c-bbf04c5ac04a on hv-cinder-maste Oct 30 01:01:09.068362 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [req-3cd48ec2-4a63-4b50-9fd6-8087b968f8ce req-562fed46-7e29-4d78-bb21-5a7c5a00ea84 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030957 Oct 30 01:01:09.068967 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 319/633] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:01:09 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:01:09.272979 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-f60f2bbb-ad78-40c5-89e9-ac753d00c1ec tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:09.273923 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-f60f2bbb-ad78-40c5-89e9-ac753d00c1ec tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:09.284936 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f60f2bbb-ad78-40c5-89e9-ac753d00c1ec tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:09.285736 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f60f2bbb-ad78-40c5-89e9-ac753d00c1ec tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:09.286430 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f60f2bbb-ad78-40c5-89e9-ac753d00c1ec tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:09.287602 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f60f2bbb-ad78-40c5-89e9-ac753d00c1ec tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:09.288388 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f60f2bbb-ad78-40c5-89e9-ac753d00c1ec tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:09.289080 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f60f2bbb-ad78-40c5-89e9-ac753d00c1ec tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:09.343612 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-ba98a113-e85f-4cc6-8b21-4a85156c6cfc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:09.344910 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-ba98a113-e85f-4cc6-8b21-4a85156c6cfc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:09.352084 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ba98a113-e85f-4cc6-8b21-4a85156c6cfc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:09.353242 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ba98a113-e85f-4cc6-8b21-4a85156c6cfc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:09.356180 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ba98a113-e85f-4cc6-8b21-4a85156c6cfc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:09.356934 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ba98a113-e85f-4cc6-8b21-4a85156c6cfc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:09.357402 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ba98a113-e85f-4cc6-8b21-4a85156c6cfc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:09.358027 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ba98a113-e85f-4cc6-8b21-4a85156c6cfc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:09.482968 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-f60f2bbb-ad78-40c5-89e9-ac753d00c1ec tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:09.492187 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-f60f2bbb-ad78-40c5-89e9-ac753d00c1ec tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1371 microversion: 2.1 time: 0.225690 Oct 30 01:01:09.497989 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 315/634] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:09 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1371 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:09.498190 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-5c48796e-9734-45d1-8bd1-bfbabc80b2d0 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:09.498912 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-5c48796e-9734-45d1-8bd1-bfbabc80b2d0 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5632075c-7062-434e-b00c-bbf04c5ac04a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:09.506223 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5c48796e-9734-45d1-8bd1-bfbabc80b2d0 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:09.506778 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5c48796e-9734-45d1-8bd1-bfbabc80b2d0 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:09.507551 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5c48796e-9734-45d1-8bd1-bfbabc80b2d0 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:09.508220 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5c48796e-9734-45d1-8bd1-bfbabc80b2d0 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:09.508668 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5c48796e-9734-45d1-8bd1-bfbabc80b2d0 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:09.509252 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5c48796e-9734-45d1-8bd1-bfbabc80b2d0 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:09.535877 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-ba98a113-e85f-4cc6-8b21-4a85156c6cfc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:09.544622 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-ba98a113-e85f-4cc6-8b21-4a85156c6cfc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:09.546101 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-ba98a113-e85f-4cc6-8b21-4a85156c6cfc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:09.547280 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-ba98a113-e85f-4cc6-8b21-4a85156c6cfc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1721 microversion: 2.20 time: 0.208270 Oct 30 01:01:09.548051 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 320/635] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:01:09 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1721 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:01:09.647449 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-5c48796e-9734-45d1-8bd1-bfbabc80b2d0 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:09.656039 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-5c48796e-9734-45d1-8bd1-bfbabc80b2d0 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a" status: 200 len: 1366 microversion: 2.1 time: 0.161664 Oct 30 01:01:09.656944 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 316/636] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:09 2022] GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a => generated 1366 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:10.516457 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-d2434f95-adbe-4bb0-b088-4d5f1420eced tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:10.517024 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-d2434f95-adbe-4bb0-b088-4d5f1420eced tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:10.524266 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d2434f95-adbe-4bb0-b088-4d5f1420eced tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:10.524748 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d2434f95-adbe-4bb0-b088-4d5f1420eced tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:10.525158 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d2434f95-adbe-4bb0-b088-4d5f1420eced tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:10.525883 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d2434f95-adbe-4bb0-b088-4d5f1420eced tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:10.526336 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d2434f95-adbe-4bb0-b088-4d5f1420eced tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:10.526795 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d2434f95-adbe-4bb0-b088-4d5f1420eced tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:10.572430 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-40c7356a-2d8d-4e2b-97c6-aad1a5df6e88 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:10.573038 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-40c7356a-2d8d-4e2b-97c6-aad1a5df6e88 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:10.580688 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-40c7356a-2d8d-4e2b-97c6-aad1a5df6e88 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:10.581233 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-40c7356a-2d8d-4e2b-97c6-aad1a5df6e88 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:10.581735 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-40c7356a-2d8d-4e2b-97c6-aad1a5df6e88 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:10.582519 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-40c7356a-2d8d-4e2b-97c6-aad1a5df6e88 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:10.583028 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-40c7356a-2d8d-4e2b-97c6-aad1a5df6e88 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:10.583503 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-40c7356a-2d8d-4e2b-97c6-aad1a5df6e88 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:10.698203 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-d2434f95-adbe-4bb0-b088-4d5f1420eced tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:10.707327 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-d2434f95-adbe-4bb0-b088-4d5f1420eced tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1371 microversion: 2.1 time: 0.195109 Oct 30 01:01:10.719791 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 321/637] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:10 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1371 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:10.719791 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-3eddbb85-31ce-4f4d-b309-984d841be14e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:10.719791 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-3eddbb85-31ce-4f4d-b309-984d841be14e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5632075c-7062-434e-b00c-bbf04c5ac04a] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:10.719791 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3eddbb85-31ce-4f4d-b309-984d841be14e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:10.720242 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3eddbb85-31ce-4f4d-b309-984d841be14e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:10.720348 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3eddbb85-31ce-4f4d-b309-984d841be14e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:10.720982 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3eddbb85-31ce-4f4d-b309-984d841be14e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:10.721426 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3eddbb85-31ce-4f4d-b309-984d841be14e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:10.721826 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3eddbb85-31ce-4f4d-b309-984d841be14e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:10.741702 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-40c7356a-2d8d-4e2b-97c6-aad1a5df6e88 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:10.749996 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-40c7356a-2d8d-4e2b-97c6-aad1a5df6e88 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:10.751418 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-40c7356a-2d8d-4e2b-97c6-aad1a5df6e88 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:10.752601 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-40c7356a-2d8d-4e2b-97c6-aad1a5df6e88 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1721 microversion: 2.20 time: 0.184154 Oct 30 01:01:10.753159 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 317/638] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:01:10 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1721 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:01:10.856050 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-3eddbb85-31ce-4f4d-b309-984d841be14e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:10.864692 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-3eddbb85-31ce-4f4d-b309-984d841be14e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a" status: 200 len: 1366 microversion: 2.1 time: 0.155501 Oct 30 01:01:10.865184 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 322/639] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:10 2022] GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a => generated 1366 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:11.736152 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-193ac5fb-f110-4cbf-8d94-d350019247f9 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:11.736753 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-193ac5fb-f110-4cbf-8d94-d350019247f9 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:11.739543 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell0:SELECT=2 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:01:11.740683 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell0:SELECT=6 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:01:11.750379 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-193ac5fb-f110-4cbf-8d94-d350019247f9 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:11.750859 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-193ac5fb-f110-4cbf-8d94-d350019247f9 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:11.751282 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-193ac5fb-f110-4cbf-8d94-d350019247f9 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:11.751992 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-193ac5fb-f110-4cbf-8d94-d350019247f9 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:11.752433 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-193ac5fb-f110-4cbf-8d94-d350019247f9 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:11.752843 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-193ac5fb-f110-4cbf-8d94-d350019247f9 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:11.777450 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-0050f392-2f82-4960-89e5-f36b71ff9940 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:11.778127 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-0050f392-2f82-4960-89e5-f36b71ff9940 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:11.780165 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell0:SELECT=2 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:01:11.780795 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell0:SELECT=14 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:01:11.787133 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0050f392-2f82-4960-89e5-f36b71ff9940 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:11.787614 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0050f392-2f82-4960-89e5-f36b71ff9940 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:11.788071 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0050f392-2f82-4960-89e5-f36b71ff9940 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:11.788760 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0050f392-2f82-4960-89e5-f36b71ff9940 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:11.789201 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0050f392-2f82-4960-89e5-f36b71ff9940 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:11.789615 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0050f392-2f82-4960-89e5-f36b71ff9940 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:11.937195 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-193ac5fb-f110-4cbf-8d94-d350019247f9 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:11.945251 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-193ac5fb-f110-4cbf-8d94-d350019247f9 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1371 microversion: 2.1 time: 0.216507 Oct 30 01:01:11.945796 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 318/640] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:11 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1371 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:11.950262 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-d7505c04-c321-489e-ab1c-8f98ca450fbe tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:11.950912 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-d7505c04-c321-489e-ab1c-8f98ca450fbe tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5632075c-7062-434e-b00c-bbf04c5ac04a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:11.956977 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d7505c04-c321-489e-ab1c-8f98ca450fbe tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:11.957475 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d7505c04-c321-489e-ab1c-8f98ca450fbe tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:11.957880 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d7505c04-c321-489e-ab1c-8f98ca450fbe tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:11.958568 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d7505c04-c321-489e-ab1c-8f98ca450fbe tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:11.959008 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d7505c04-c321-489e-ab1c-8f98ca450fbe tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:11.959423 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d7505c04-c321-489e-ab1c-8f98ca450fbe tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:11.963508 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell1:SELECT=100 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:01:11.964840 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-0050f392-2f82-4960-89e5-f36b71ff9940 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:11.973479 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-0050f392-2f82-4960-89e5-f36b71ff9940 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:11.974903 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-0050f392-2f82-4960-89e5-f36b71ff9940 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:11.976122 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-0050f392-2f82-4960-89e5-f36b71ff9940 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1721 microversion: 2.20 time: 0.202504 Oct 30 01:01:11.976642 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 323/641] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:01:11 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1721 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 30 01:01:12.093806 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-d7505c04-c321-489e-ab1c-8f98ca450fbe tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:12.101587 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-d7505c04-c321-489e-ab1c-8f98ca450fbe tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a" status: 200 len: 1366 microversion: 2.1 time: 0.154840 Oct 30 01:01:12.102165 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 319/642] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:11 2022] GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a => generated 1366 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:12.972370 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-a1b5ba47-0e02-48fc-9bc5-296ad89f240a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:12.973304 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-a1b5ba47-0e02-48fc-9bc5-296ad89f240a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:12.982727 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a1b5ba47-0e02-48fc-9bc5-296ad89f240a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:12.983216 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a1b5ba47-0e02-48fc-9bc5-296ad89f240a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:12.983652 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a1b5ba47-0e02-48fc-9bc5-296ad89f240a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:12.984383 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a1b5ba47-0e02-48fc-9bc5-296ad89f240a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:12.984837 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a1b5ba47-0e02-48fc-9bc5-296ad89f240a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:12.985261 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a1b5ba47-0e02-48fc-9bc5-296ad89f240a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:13.001789 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-f14a298e-6bf3-4606-9ddd-5235a424e955 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:13.002514 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-f14a298e-6bf3-4606-9ddd-5235a424e955 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:13.010382 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f14a298e-6bf3-4606-9ddd-5235a424e955 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:13.010937 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f14a298e-6bf3-4606-9ddd-5235a424e955 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:13.011428 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f14a298e-6bf3-4606-9ddd-5235a424e955 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:13.012292 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f14a298e-6bf3-4606-9ddd-5235a424e955 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:13.012946 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f14a298e-6bf3-4606-9ddd-5235a424e955 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:13.013366 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f14a298e-6bf3-4606-9ddd-5235a424e955 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:13.201583 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-f14a298e-6bf3-4606-9ddd-5235a424e955 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:13.201975 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-a1b5ba47-0e02-48fc-9bc5-296ad89f240a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:13.210723 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-a1b5ba47-0e02-48fc-9bc5-296ad89f240a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1371 microversion: 2.1 time: 0.244477 Oct 30 01:01:13.218498 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-f14a298e-6bf3-4606-9ddd-5235a424e955 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:13.221340 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-f14a298e-6bf3-4606-9ddd-5235a424e955 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:13.223909 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 324/643] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:12 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1371 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:13.225109 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-f14a298e-6bf3-4606-9ddd-5235a424e955 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1724 microversion: 2.20 time: 0.228464 Oct 30 01:01:13.225769 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 320/644] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:01:12 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1724 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:01:13.233964 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-0e804f30-a318-4bf4-aae4-38fcac935552 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:13.234600 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-0e804f30-a318-4bf4-aae4-38fcac935552 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5632075c-7062-434e-b00c-bbf04c5ac04a] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:13.242242 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0e804f30-a318-4bf4-aae4-38fcac935552 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:13.242897 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0e804f30-a318-4bf4-aae4-38fcac935552 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:13.243338 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0e804f30-a318-4bf4-aae4-38fcac935552 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:13.244177 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0e804f30-a318-4bf4-aae4-38fcac935552 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:13.244734 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0e804f30-a318-4bf4-aae4-38fcac935552 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:13.245191 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0e804f30-a318-4bf4-aae4-38fcac935552 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:13.251391 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-f25b2c84-da56-4078-baa3-b43fac595265 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "44b5cbd4-240a-46bd-bc05-26ae29cce184"}} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:01:13.255775 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-f25b2c84-da56-4078-baa3-b43fac595265 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:13.265821 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f25b2c84-da56-4078-baa3-b43fac595265 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:13.266770 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f25b2c84-da56-4078-baa3-b43fac595265 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:13.267538 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f25b2c84-da56-4078-baa3-b43fac595265 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:13.269835 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f25b2c84-da56-4078-baa3-b43fac595265 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:13.271224 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f25b2c84-da56-4078-baa3-b43fac595265 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:13.272262 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f25b2c84-da56-4078-baa3-b43fac595265 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:13.392088 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-0e804f30-a318-4bf4-aae4-38fcac935552 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:13.399875 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-0e804f30-a318-4bf4-aae4-38fcac935552 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a" status: 200 len: 1366 microversion: 2.1 time: 0.174690 Oct 30 01:01:13.400478 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 325/645] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:13 2022] GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a => generated 1366 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:13.581870 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-f25b2c84-da56-4078-baa3-b43fac595265 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "POST /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.335780 Oct 30 01:01:13.582554 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 321/646] 10.222.0.16 () {62 vars in 1364 bytes} [Sun Oct 30 01:01:13 2022] POST /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f/os-volume_attachments => generated 188 bytes in 337 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 30 01:01:13.708650 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-d96afaff-4f71-49ec-ada0-bbf69bf27ef0 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:01:13.709257 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-d96afaff-4f71-49ec-ada0-bbf69bf27ef0 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:13.715846 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d96afaff-4f71-49ec-ada0-bbf69bf27ef0 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:13.716325 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d96afaff-4f71-49ec-ada0-bbf69bf27ef0 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:13.716775 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d96afaff-4f71-49ec-ada0-bbf69bf27ef0 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:13.717468 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d96afaff-4f71-49ec-ada0-bbf69bf27ef0 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:13.717921 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d96afaff-4f71-49ec-ada0-bbf69bf27ef0 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:13.718344 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d96afaff-4f71-49ec-ada0-bbf69bf27ef0 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:13.815502 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-d96afaff-4f71-49ec-ada0-bbf69bf27ef0 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=118961) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Oct 30 01:01:13.890334 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-d96afaff-4f71-49ec-ada0-bbf69bf27ef0 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "POST /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f/action" status: 202 len: 0 microversion: 2.20 time: 0.185591 Oct 30 01:01:13.890960 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 326/647] 10.222.0.16 () {62 vars in 1319 bytes} [Sun Oct 30 01:01:13 2022] POST /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f/action => generated 0 bytes in 187 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 30 01:01:13.899167 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-b23499c8-e301-4174-9dc1-189a458055c1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:13.899815 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-b23499c8-e301-4174-9dc1-189a458055c1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:13.906425 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b23499c8-e301-4174-9dc1-189a458055c1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:13.906872 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b23499c8-e301-4174-9dc1-189a458055c1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:13.907323 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b23499c8-e301-4174-9dc1-189a458055c1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:13.908047 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b23499c8-e301-4174-9dc1-189a458055c1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:13.908484 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b23499c8-e301-4174-9dc1-189a458055c1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:13.908880 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b23499c8-e301-4174-9dc1-189a458055c1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:14.043598 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-b23499c8-e301-4174-9dc1-189a458055c1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:14.052643 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-b23499c8-e301-4174-9dc1-189a458055c1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:14.054071 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-b23499c8-e301-4174-9dc1-189a458055c1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:14.055236 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-b23499c8-e301-4174-9dc1-189a458055c1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1810 microversion: 2.20 time: 0.159963 Oct 30 01:01:14.055800 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 322/648] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:01:13 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1810 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:01:14.235283 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-ec7f6365-5876-410b-9c99-3ef24004e923 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:14.235930 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-ec7f6365-5876-410b-9c99-3ef24004e923 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:14.242513 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ec7f6365-5876-410b-9c99-3ef24004e923 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:14.243009 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ec7f6365-5876-410b-9c99-3ef24004e923 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:14.243415 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ec7f6365-5876-410b-9c99-3ef24004e923 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:14.244155 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ec7f6365-5876-410b-9c99-3ef24004e923 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:14.244628 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ec7f6365-5876-410b-9c99-3ef24004e923 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:14.245076 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ec7f6365-5876-410b-9c99-3ef24004e923 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:14.381476 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-ec7f6365-5876-410b-9c99-3ef24004e923 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:14.389587 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-ec7f6365-5876-410b-9c99-3ef24004e923 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1371 microversion: 2.1 time: 0.158844 Oct 30 01:01:14.390246 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 327/649] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:14 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1371 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:14.422730 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-9d3fd834-9978-4153-b063-f17d9ba4ff21 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:14.423340 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-9d3fd834-9978-4153-b063-f17d9ba4ff21 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5632075c-7062-434e-b00c-bbf04c5ac04a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:14.429337 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9d3fd834-9978-4153-b063-f17d9ba4ff21 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:14.429866 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9d3fd834-9978-4153-b063-f17d9ba4ff21 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:14.430301 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9d3fd834-9978-4153-b063-f17d9ba4ff21 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:14.431121 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9d3fd834-9978-4153-b063-f17d9ba4ff21 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:14.431412 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9d3fd834-9978-4153-b063-f17d9ba4ff21 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:14.431854 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9d3fd834-9978-4153-b063-f17d9ba4ff21 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:14.564145 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-9d3fd834-9978-4153-b063-f17d9ba4ff21 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:14.572013 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-9d3fd834-9978-4153-b063-f17d9ba4ff21 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a" status: 200 len: 1366 microversion: 2.1 time: 0.153258 Oct 30 01:01:14.572588 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 323/650] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:14 2022] GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a => generated 1366 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:15.085985 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-5a11b125-ba3d-4224-abe4-d39629fa221d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:15.087077 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-5a11b125-ba3d-4224-abe4-d39629fa221d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:15.095384 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5a11b125-ba3d-4224-abe4-d39629fa221d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:15.095901 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5a11b125-ba3d-4224-abe4-d39629fa221d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:15.096341 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5a11b125-ba3d-4224-abe4-d39629fa221d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:15.097026 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5a11b125-ba3d-4224-abe4-d39629fa221d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:15.097471 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5a11b125-ba3d-4224-abe4-d39629fa221d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:15.097885 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5a11b125-ba3d-4224-abe4-d39629fa221d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:15.233476 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-5a11b125-ba3d-4224-abe4-d39629fa221d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:15.242908 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-5a11b125-ba3d-4224-abe4-d39629fa221d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:15.244365 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-5a11b125-ba3d-4224-abe4-d39629fa221d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:15.245552 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-5a11b125-ba3d-4224-abe4-d39629fa221d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1810 microversion: 2.20 time: 0.166202 Oct 30 01:01:15.246106 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 328/651] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:01:15 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1810 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:01:15.417442 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-2d218164-8133-4304-a01e-a832f7f8ac8a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:15.418413 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-2d218164-8133-4304-a01e-a832f7f8ac8a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:15.421141 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:01:15.426647 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2d218164-8133-4304-a01e-a832f7f8ac8a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:15.427123 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2d218164-8133-4304-a01e-a832f7f8ac8a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:15.427561 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2d218164-8133-4304-a01e-a832f7f8ac8a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:15.428275 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2d218164-8133-4304-a01e-a832f7f8ac8a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:15.428703 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2d218164-8133-4304-a01e-a832f7f8ac8a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:15.429124 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2d218164-8133-4304-a01e-a832f7f8ac8a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:15.563089 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-2d218164-8133-4304-a01e-a832f7f8ac8a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:15.571520 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-2d218164-8133-4304-a01e-a832f7f8ac8a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1371 microversion: 2.1 time: 0.160836 Oct 30 01:01:15.572103 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 324/652] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:15 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1371 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:15.595291 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-c51581ea-6fbc-4f15-8ddf-896d252ee0c2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:15.595948 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-c51581ea-6fbc-4f15-8ddf-896d252ee0c2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5632075c-7062-434e-b00c-bbf04c5ac04a] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:15.602509 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c51581ea-6fbc-4f15-8ddf-896d252ee0c2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:15.603222 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c51581ea-6fbc-4f15-8ddf-896d252ee0c2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:15.603632 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c51581ea-6fbc-4f15-8ddf-896d252ee0c2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:15.604380 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c51581ea-6fbc-4f15-8ddf-896d252ee0c2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:15.604942 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c51581ea-6fbc-4f15-8ddf-896d252ee0c2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:15.605364 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c51581ea-6fbc-4f15-8ddf-896d252ee0c2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:15.735877 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-c51581ea-6fbc-4f15-8ddf-896d252ee0c2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:15.741329 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:01:15.746762 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-c51581ea-6fbc-4f15-8ddf-896d252ee0c2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a" status: 200 len: 1366 microversion: 2.1 time: 0.155605 Oct 30 01:01:15.747364 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 329/653] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:15 2022] GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a => generated 1366 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:16.274781 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-c3a1a48d-1447-4bc5-9c1b-a5243b513199 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:16.275875 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-c3a1a48d-1447-4bc5-9c1b-a5243b513199 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:16.283265 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c3a1a48d-1447-4bc5-9c1b-a5243b513199 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:16.283754 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c3a1a48d-1447-4bc5-9c1b-a5243b513199 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:16.284190 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c3a1a48d-1447-4bc5-9c1b-a5243b513199 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:16.284873 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c3a1a48d-1447-4bc5-9c1b-a5243b513199 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:16.285315 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c3a1a48d-1447-4bc5-9c1b-a5243b513199 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:16.285731 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c3a1a48d-1447-4bc5-9c1b-a5243b513199 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:16.415829 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-c3a1a48d-1447-4bc5-9c1b-a5243b513199 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:16.429648 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-c3a1a48d-1447-4bc5-9c1b-a5243b513199 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:16.431614 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-c3a1a48d-1447-4bc5-9c1b-a5243b513199 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:16.433334 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-c3a1a48d-1447-4bc5-9c1b-a5243b513199 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1810 microversion: 2.20 time: 0.164991 Oct 30 01:01:16.434119 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 325/654] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:01:16 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1810 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:01:16.600291 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-45389c72-37a3-403f-9e27-33c49bb151fc tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:16.601148 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-45389c72-37a3-403f-9e27-33c49bb151fc tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:16.611244 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-45389c72-37a3-403f-9e27-33c49bb151fc tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:16.611782 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-45389c72-37a3-403f-9e27-33c49bb151fc tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:16.612224 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-45389c72-37a3-403f-9e27-33c49bb151fc tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:16.612923 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-45389c72-37a3-403f-9e27-33c49bb151fc tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:16.613377 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-45389c72-37a3-403f-9e27-33c49bb151fc tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:16.613801 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-45389c72-37a3-403f-9e27-33c49bb151fc tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:16.758191 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-45389c72-37a3-403f-9e27-33c49bb151fc tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:16.766952 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-45389c72-37a3-403f-9e27-33c49bb151fc tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1371 microversion: 2.1 time: 0.173244 Oct 30 01:01:16.767736 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 330/655] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:16 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1371 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:16.769634 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-6f6210fd-f97e-4f25-b109-9736b7656e9b tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:16.770239 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-6f6210fd-f97e-4f25-b109-9736b7656e9b tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5632075c-7062-434e-b00c-bbf04c5ac04a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:16.776328 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6f6210fd-f97e-4f25-b109-9736b7656e9b tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:16.776795 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6f6210fd-f97e-4f25-b109-9736b7656e9b tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:16.777218 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6f6210fd-f97e-4f25-b109-9736b7656e9b tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:16.777917 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6f6210fd-f97e-4f25-b109-9736b7656e9b tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:16.778355 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6f6210fd-f97e-4f25-b109-9736b7656e9b tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:16.778771 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6f6210fd-f97e-4f25-b109-9736b7656e9b tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:16.912765 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-6f6210fd-f97e-4f25-b109-9736b7656e9b tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:16.924622 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-6f6210fd-f97e-4f25-b109-9736b7656e9b tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a" status: 200 len: 1366 microversion: 2.1 time: 0.158797 Oct 30 01:01:16.925402 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 326/656] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:16 2022] GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a => generated 1366 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 01:01:17.464298 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-8795e77d-0ce0-4c17-8844-64de64cf3eca tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:17.465409 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-8795e77d-0ce0-4c17-8844-64de64cf3eca tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:17.476056 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8795e77d-0ce0-4c17-8844-64de64cf3eca tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:17.476851 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8795e77d-0ce0-4c17-8844-64de64cf3eca tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:17.477567 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8795e77d-0ce0-4c17-8844-64de64cf3eca tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:17.478702 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8795e77d-0ce0-4c17-8844-64de64cf3eca tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:17.479449 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8795e77d-0ce0-4c17-8844-64de64cf3eca tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:17.480181 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8795e77d-0ce0-4c17-8844-64de64cf3eca tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:17.615666 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-8795e77d-0ce0-4c17-8844-64de64cf3eca tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:17.624838 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-8795e77d-0ce0-4c17-8844-64de64cf3eca tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:17.626315 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-8795e77d-0ce0-4c17-8844-64de64cf3eca tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:17.627485 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-8795e77d-0ce0-4c17-8844-64de64cf3eca tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1810 microversion: 2.20 time: 0.169956 Oct 30 01:01:17.628080 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 331/657] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:01:17 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1810 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:01:17.793904 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-dceabc1e-d84b-42dd-be6e-3ec679e6b909 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:17.794954 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-dceabc1e-d84b-42dd-be6e-3ec679e6b909 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:17.805871 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dceabc1e-d84b-42dd-be6e-3ec679e6b909 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:17.806714 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dceabc1e-d84b-42dd-be6e-3ec679e6b909 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:17.807391 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dceabc1e-d84b-42dd-be6e-3ec679e6b909 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:17.808605 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dceabc1e-d84b-42dd-be6e-3ec679e6b909 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:17.809364 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dceabc1e-d84b-42dd-be6e-3ec679e6b909 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:17.810078 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dceabc1e-d84b-42dd-be6e-3ec679e6b909 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:17.941810 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-dceabc1e-d84b-42dd-be6e-3ec679e6b909 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:17.949604 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-dceabc1e-d84b-42dd-be6e-3ec679e6b909 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1371 microversion: 2.1 time: 0.162355 Oct 30 01:01:17.950183 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 327/658] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:17 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1371 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:17.951746 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-b5cc86b4-31bf-4cc4-a8c7-ba7f644ea116 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:17.952498 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-b5cc86b4-31bf-4cc4-a8c7-ba7f644ea116 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5632075c-7062-434e-b00c-bbf04c5ac04a] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:17.959121 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b5cc86b4-31bf-4cc4-a8c7-ba7f644ea116 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:17.959605 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b5cc86b4-31bf-4cc4-a8c7-ba7f644ea116 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:17.960052 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b5cc86b4-31bf-4cc4-a8c7-ba7f644ea116 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:17.960742 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b5cc86b4-31bf-4cc4-a8c7-ba7f644ea116 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:17.961188 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b5cc86b4-31bf-4cc4-a8c7-ba7f644ea116 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:17.961608 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b5cc86b4-31bf-4cc4-a8c7-ba7f644ea116 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:18.095942 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-b5cc86b4-31bf-4cc4-a8c7-ba7f644ea116 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:18.103951 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-b5cc86b4-31bf-4cc4-a8c7-ba7f644ea116 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a" status: 200 len: 1366 microversion: 2.1 time: 0.156152 Oct 30 01:01:18.104538 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 332/659] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:17 2022] GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a => generated 1366 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:18.666685 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-8e08ada3-bbfe-4e97-a0db-af0c83e558ae tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:18.667544 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-8e08ada3-bbfe-4e97-a0db-af0c83e558ae tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:18.677439 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8e08ada3-bbfe-4e97-a0db-af0c83e558ae tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:18.678229 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8e08ada3-bbfe-4e97-a0db-af0c83e558ae tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:18.678937 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8e08ada3-bbfe-4e97-a0db-af0c83e558ae tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:18.680106 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8e08ada3-bbfe-4e97-a0db-af0c83e558ae tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:18.680880 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8e08ada3-bbfe-4e97-a0db-af0c83e558ae tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:18.681562 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8e08ada3-bbfe-4e97-a0db-af0c83e558ae tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:18.819016 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-8e08ada3-bbfe-4e97-a0db-af0c83e558ae tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:18.828294 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-8e08ada3-bbfe-4e97-a0db-af0c83e558ae tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:18.829759 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-8e08ada3-bbfe-4e97-a0db-af0c83e558ae tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:18.830914 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-8e08ada3-bbfe-4e97-a0db-af0c83e558ae tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1810 microversion: 2.20 time: 0.170954 Oct 30 01:01:18.831477 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 328/660] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:01:18 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1810 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:01:18.972481 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-5e5dc87b-9de5-41f6-8b50-1a568a506de9 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:18.973122 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-5e5dc87b-9de5-41f6-8b50-1a568a506de9 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:18.979982 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5e5dc87b-9de5-41f6-8b50-1a568a506de9 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:18.980470 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5e5dc87b-9de5-41f6-8b50-1a568a506de9 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:18.980907 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5e5dc87b-9de5-41f6-8b50-1a568a506de9 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:18.981602 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5e5dc87b-9de5-41f6-8b50-1a568a506de9 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:18.982056 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5e5dc87b-9de5-41f6-8b50-1a568a506de9 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:18.982476 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5e5dc87b-9de5-41f6-8b50-1a568a506de9 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:19.111648 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-5e5dc87b-9de5-41f6-8b50-1a568a506de9 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:19.122476 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-5e5dc87b-9de5-41f6-8b50-1a568a506de9 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1371 microversion: 2.1 time: 0.153915 Oct 30 01:01:19.123064 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 333/661] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:18 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1371 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:19.126571 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-f68bed53-f106-44b8-932c-b732c4070506 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:19.127199 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-f68bed53-f106-44b8-932c-b732c4070506 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5632075c-7062-434e-b00c-bbf04c5ac04a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:19.133289 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f68bed53-f106-44b8-932c-b732c4070506 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:19.133757 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f68bed53-f106-44b8-932c-b732c4070506 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:19.134179 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f68bed53-f106-44b8-932c-b732c4070506 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:19.134883 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f68bed53-f106-44b8-932c-b732c4070506 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:19.135319 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f68bed53-f106-44b8-932c-b732c4070506 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:19.135754 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f68bed53-f106-44b8-932c-b732c4070506 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:19.270611 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-f68bed53-f106-44b8-932c-b732c4070506 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:19.278406 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-f68bed53-f106-44b8-932c-b732c4070506 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a" status: 200 len: 1366 microversion: 2.1 time: 0.155677 Oct 30 01:01:19.278973 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 329/662] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:19 2022] GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a => generated 1366 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 01:01:19.865136 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-bd275214-9e52-4f90-9ca1-b98a1c5659b4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:19.865746 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-bd275214-9e52-4f90-9ca1-b98a1c5659b4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:19.872218 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bd275214-9e52-4f90-9ca1-b98a1c5659b4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:19.872695 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bd275214-9e52-4f90-9ca1-b98a1c5659b4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:19.873126 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bd275214-9e52-4f90-9ca1-b98a1c5659b4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:19.873864 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bd275214-9e52-4f90-9ca1-b98a1c5659b4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:19.874317 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bd275214-9e52-4f90-9ca1-b98a1c5659b4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:19.874733 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bd275214-9e52-4f90-9ca1-b98a1c5659b4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:20.005311 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-bd275214-9e52-4f90-9ca1-b98a1c5659b4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:20.014030 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-bd275214-9e52-4f90-9ca1-b98a1c5659b4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:20.015498 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-bd275214-9e52-4f90-9ca1-b98a1c5659b4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:20.016693 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-bd275214-9e52-4f90-9ca1-b98a1c5659b4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1810 microversion: 2.20 time: 0.155619 Oct 30 01:01:20.017258 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 334/663] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:01:19 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1810 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:01:20.149888 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-820673fd-720d-405a-b28e-936ae980eff2 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:20.150877 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-820673fd-720d-405a-b28e-936ae980eff2 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:20.160195 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-820673fd-720d-405a-b28e-936ae980eff2 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:20.160701 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-820673fd-720d-405a-b28e-936ae980eff2 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:20.161124 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-820673fd-720d-405a-b28e-936ae980eff2 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:20.161808 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-820673fd-720d-405a-b28e-936ae980eff2 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:20.162248 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-820673fd-720d-405a-b28e-936ae980eff2 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:20.162660 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-820673fd-720d-405a-b28e-936ae980eff2 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:20.301525 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-04c68485-964a-43be-ab37-65720a516d77 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:20.302155 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-04c68485-964a-43be-ab37-65720a516d77 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5632075c-7062-434e-b00c-bbf04c5ac04a] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:20.305028 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_api:SELECT=95,nova_api:INSERT=4,nova_api:UPDATE=1 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:01:20.306839 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-820673fd-720d-405a-b28e-936ae980eff2 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:20.308989 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-04c68485-964a-43be-ab37-65720a516d77 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:20.309471 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-04c68485-964a-43be-ab37-65720a516d77 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:20.309898 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-04c68485-964a-43be-ab37-65720a516d77 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:20.310864 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-04c68485-964a-43be-ab37-65720a516d77 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:20.311314 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-04c68485-964a-43be-ab37-65720a516d77 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:20.311881 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-04c68485-964a-43be-ab37-65720a516d77 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:20.315230 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-820673fd-720d-405a-b28e-936ae980eff2 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1371 microversion: 2.1 time: 0.171990 Oct 30 01:01:20.315819 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 330/664] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:20 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1371 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:20.453344 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-04c68485-964a-43be-ab37-65720a516d77 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:20.461814 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-04c68485-964a-43be-ab37-65720a516d77 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a" status: 200 len: 1366 microversion: 2.1 time: 0.164337 Oct 30 01:01:20.462395 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 335/665] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:20 2022] GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a => generated 1366 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:21.042502 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-c9c8b598-3f32-4174-b687-1626ea43130c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:21.043110 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-c9c8b598-3f32-4174-b687-1626ea43130c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:21.050004 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c9c8b598-3f32-4174-b687-1626ea43130c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:21.050565 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c9c8b598-3f32-4174-b687-1626ea43130c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:21.051079 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c9c8b598-3f32-4174-b687-1626ea43130c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:21.051883 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c9c8b598-3f32-4174-b687-1626ea43130c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:21.052395 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c9c8b598-3f32-4174-b687-1626ea43130c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:21.052874 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c9c8b598-3f32-4174-b687-1626ea43130c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:21.189861 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-c9c8b598-3f32-4174-b687-1626ea43130c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:21.198941 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-c9c8b598-3f32-4174-b687-1626ea43130c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:21.200762 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-c9c8b598-3f32-4174-b687-1626ea43130c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:21.202341 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-c9c8b598-3f32-4174-b687-1626ea43130c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1810 microversion: 2.20 time: 0.163746 Oct 30 01:01:21.202894 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 331/666] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:01:21 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1810 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:01:21.346206 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-7a6e82c2-b60a-4025-a59e-aab1138edf16 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:21.346802 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-7a6e82c2-b60a-4025-a59e-aab1138edf16 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:21.353807 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7a6e82c2-b60a-4025-a59e-aab1138edf16 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:21.354286 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7a6e82c2-b60a-4025-a59e-aab1138edf16 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:21.354715 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7a6e82c2-b60a-4025-a59e-aab1138edf16 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:21.355416 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7a6e82c2-b60a-4025-a59e-aab1138edf16 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:21.355943 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7a6e82c2-b60a-4025-a59e-aab1138edf16 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:21.356310 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7a6e82c2-b60a-4025-a59e-aab1138edf16 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:21.485855 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-d6ef46dc-4a66-44bd-9697-6503ec3100ae tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:21.486461 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-d6ef46dc-4a66-44bd-9697-6503ec3100ae tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5632075c-7062-434e-b00c-bbf04c5ac04a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:21.492435 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d6ef46dc-4a66-44bd-9697-6503ec3100ae tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:21.492912 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d6ef46dc-4a66-44bd-9697-6503ec3100ae tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:21.493332 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d6ef46dc-4a66-44bd-9697-6503ec3100ae tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:21.494008 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d6ef46dc-4a66-44bd-9697-6503ec3100ae tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:21.494482 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d6ef46dc-4a66-44bd-9697-6503ec3100ae tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:21.494940 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d6ef46dc-4a66-44bd-9697-6503ec3100ae tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:21.497421 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-7a6e82c2-b60a-4025-a59e-aab1138edf16 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:21.505896 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-7a6e82c2-b60a-4025-a59e-aab1138edf16 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1371 microversion: 2.1 time: 0.165890 Oct 30 01:01:21.506475 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 336/667] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:21 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1371 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:21.628628 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-d6ef46dc-4a66-44bd-9697-6503ec3100ae tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:21.636997 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-d6ef46dc-4a66-44bd-9697-6503ec3100ae tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a" status: 200 len: 1366 microversion: 2.1 time: 0.155650 Oct 30 01:01:21.637587 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 332/668] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:21 2022] GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a => generated 1366 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:22.234482 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-1ce30e48-35a3-4752-a8b4-3e44e202d962 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:22.235114 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-1ce30e48-35a3-4752-a8b4-3e44e202d962 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:22.242023 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1ce30e48-35a3-4752-a8b4-3e44e202d962 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:22.242526 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1ce30e48-35a3-4752-a8b4-3e44e202d962 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:22.242992 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1ce30e48-35a3-4752-a8b4-3e44e202d962 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:22.243731 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1ce30e48-35a3-4752-a8b4-3e44e202d962 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:22.244205 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1ce30e48-35a3-4752-a8b4-3e44e202d962 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:22.244642 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1ce30e48-35a3-4752-a8b4-3e44e202d962 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:22.386981 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-1ce30e48-35a3-4752-a8b4-3e44e202d962 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:22.395948 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-1ce30e48-35a3-4752-a8b4-3e44e202d962 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:22.397371 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-1ce30e48-35a3-4752-a8b4-3e44e202d962 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:22.398528 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-1ce30e48-35a3-4752-a8b4-3e44e202d962 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1810 microversion: 2.20 time: 0.168212 Oct 30 01:01:22.399116 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 337/669] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:01:22 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1810 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:01:22.533192 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-4c3e9242-c6ad-4d79-b01c-f83f04767435 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:22.534150 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-4c3e9242-c6ad-4d79-b01c-f83f04767435 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:22.545467 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4c3e9242-c6ad-4d79-b01c-f83f04767435 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:22.546255 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4c3e9242-c6ad-4d79-b01c-f83f04767435 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:22.547002 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4c3e9242-c6ad-4d79-b01c-f83f04767435 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:22.548128 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4c3e9242-c6ad-4d79-b01c-f83f04767435 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:22.548869 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4c3e9242-c6ad-4d79-b01c-f83f04767435 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:22.549833 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4c3e9242-c6ad-4d79-b01c-f83f04767435 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:22.553791 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell1:SELECT=93,nova_cell1:INSERT=3,nova_cell1:UPDATE=4 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:01:22.660535 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-98ae22ec-264c-406e-8370-bfcb988ea578 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:22.661354 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-98ae22ec-264c-406e-8370-bfcb988ea578 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5632075c-7062-434e-b00c-bbf04c5ac04a] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:22.667589 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-98ae22ec-264c-406e-8370-bfcb988ea578 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:22.668120 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-98ae22ec-264c-406e-8370-bfcb988ea578 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:22.668537 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-98ae22ec-264c-406e-8370-bfcb988ea578 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:22.669237 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-98ae22ec-264c-406e-8370-bfcb988ea578 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:22.669684 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-98ae22ec-264c-406e-8370-bfcb988ea578 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:22.670111 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-98ae22ec-264c-406e-8370-bfcb988ea578 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:22.697456 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-4c3e9242-c6ad-4d79-b01c-f83f04767435 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:22.706039 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-4c3e9242-c6ad-4d79-b01c-f83f04767435 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1371 microversion: 2.1 time: 0.179207 Oct 30 01:01:22.706634 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 333/670] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:22 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1371 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:22.811318 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-98ae22ec-264c-406e-8370-bfcb988ea578 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:22.820742 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-98ae22ec-264c-406e-8370-bfcb988ea578 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a" status: 200 len: 1366 microversion: 2.1 time: 0.164339 Oct 30 01:01:22.821315 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 338/671] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:22 2022] GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a => generated 1366 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:23.424768 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-7aa3cbca-b5c4-425b-8b97-2247791bf2de tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:23.425296 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-7aa3cbca-b5c4-425b-8b97-2247791bf2de tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:23.431676 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7aa3cbca-b5c4-425b-8b97-2247791bf2de tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:23.432193 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7aa3cbca-b5c4-425b-8b97-2247791bf2de tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:23.432620 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7aa3cbca-b5c4-425b-8b97-2247791bf2de tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:23.433305 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7aa3cbca-b5c4-425b-8b97-2247791bf2de tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:23.433753 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7aa3cbca-b5c4-425b-8b97-2247791bf2de tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:23.434195 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7aa3cbca-b5c4-425b-8b97-2247791bf2de tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:23.561692 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-7aa3cbca-b5c4-425b-8b97-2247791bf2de tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:23.570924 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-7aa3cbca-b5c4-425b-8b97-2247791bf2de tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:23.572388 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-7aa3cbca-b5c4-425b-8b97-2247791bf2de tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:23.573530 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-7aa3cbca-b5c4-425b-8b97-2247791bf2de tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1810 microversion: 2.20 time: 0.152990 Oct 30 01:01:23.574089 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 334/672] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:01:23 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1810 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:01:23.733275 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-22dcffb5-a383-404c-a3a8-64f5dfc51ee5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:23.734153 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-22dcffb5-a383-404c-a3a8-64f5dfc51ee5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:23.745218 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-22dcffb5-a383-404c-a3a8-64f5dfc51ee5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:23.746061 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-22dcffb5-a383-404c-a3a8-64f5dfc51ee5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:23.746909 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-22dcffb5-a383-404c-a3a8-64f5dfc51ee5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:23.747985 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-22dcffb5-a383-404c-a3a8-64f5dfc51ee5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:23.748731 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-22dcffb5-a383-404c-a3a8-64f5dfc51ee5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:23.749431 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-22dcffb5-a383-404c-a3a8-64f5dfc51ee5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:23.844589 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-f388f322-b5c0-4958-b2b3-ba426cdd41a1 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:23.845208 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-f388f322-b5c0-4958-b2b3-ba426cdd41a1 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5632075c-7062-434e-b00c-bbf04c5ac04a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:23.852266 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f388f322-b5c0-4958-b2b3-ba426cdd41a1 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:23.852771 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f388f322-b5c0-4958-b2b3-ba426cdd41a1 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:23.853194 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f388f322-b5c0-4958-b2b3-ba426cdd41a1 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:23.853901 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f388f322-b5c0-4958-b2b3-ba426cdd41a1 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:23.854350 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f388f322-b5c0-4958-b2b3-ba426cdd41a1 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:23.854948 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f388f322-b5c0-4958-b2b3-ba426cdd41a1 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:23.888073 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-22dcffb5-a383-404c-a3a8-64f5dfc51ee5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:23.896393 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-22dcffb5-a383-404c-a3a8-64f5dfc51ee5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1371 microversion: 2.1 time: 0.169889 Oct 30 01:01:23.897119 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 339/673] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:23 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1371 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:23.988538 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-f388f322-b5c0-4958-b2b3-ba426cdd41a1 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:23.996463 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-f388f322-b5c0-4958-b2b3-ba426cdd41a1 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a" status: 200 len: 1366 microversion: 2.1 time: 0.155932 Oct 30 01:01:23.997004 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 335/674] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:23 2022] GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a => generated 1366 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:24.601514 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-281b0f52-609c-4d41-a6fb-e12ce6ef2c59 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:24.602075 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-281b0f52-609c-4d41-a6fb-e12ce6ef2c59 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:24.609164 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-281b0f52-609c-4d41-a6fb-e12ce6ef2c59 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:24.609687 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-281b0f52-609c-4d41-a6fb-e12ce6ef2c59 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:24.610158 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-281b0f52-609c-4d41-a6fb-e12ce6ef2c59 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:24.610914 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-281b0f52-609c-4d41-a6fb-e12ce6ef2c59 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:24.611388 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-281b0f52-609c-4d41-a6fb-e12ce6ef2c59 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:24.611903 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-281b0f52-609c-4d41-a6fb-e12ce6ef2c59 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:24.746188 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-281b0f52-609c-4d41-a6fb-e12ce6ef2c59 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:24.759978 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-281b0f52-609c-4d41-a6fb-e12ce6ef2c59 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:24.762223 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-281b0f52-609c-4d41-a6fb-e12ce6ef2c59 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:24.763610 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-281b0f52-609c-4d41-a6fb-e12ce6ef2c59 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1810 microversion: 2.20 time: 0.166245 Oct 30 01:01:24.764610 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 340/675] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:01:24 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1810 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:01:24.924250 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-0b8a3704-11b8-4993-930d-0f237ba60d56 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:24.925072 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-0b8a3704-11b8-4993-930d-0f237ba60d56 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:24.932558 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0b8a3704-11b8-4993-930d-0f237ba60d56 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:24.933064 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0b8a3704-11b8-4993-930d-0f237ba60d56 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:24.933513 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0b8a3704-11b8-4993-930d-0f237ba60d56 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:24.934241 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0b8a3704-11b8-4993-930d-0f237ba60d56 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:24.934712 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0b8a3704-11b8-4993-930d-0f237ba60d56 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:24.935152 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0b8a3704-11b8-4993-930d-0f237ba60d56 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:25.019702 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-e64d4cbc-fc35-46f2-9527-c1956e65cf45 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:25.020341 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-e64d4cbc-fc35-46f2-9527-c1956e65cf45 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5632075c-7062-434e-b00c-bbf04c5ac04a] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:25.026764 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e64d4cbc-fc35-46f2-9527-c1956e65cf45 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:25.027257 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e64d4cbc-fc35-46f2-9527-c1956e65cf45 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:25.027693 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e64d4cbc-fc35-46f2-9527-c1956e65cf45 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:25.028426 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e64d4cbc-fc35-46f2-9527-c1956e65cf45 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:25.028854 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e64d4cbc-fc35-46f2-9527-c1956e65cf45 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:25.029308 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e64d4cbc-fc35-46f2-9527-c1956e65cf45 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:25.079017 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-0b8a3704-11b8-4993-930d-0f237ba60d56 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:25.087087 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-0b8a3704-11b8-4993-930d-0f237ba60d56 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1371 microversion: 2.1 time: 0.169451 Oct 30 01:01:25.087663 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 336/676] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:24 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1371 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:25.165667 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-e64d4cbc-fc35-46f2-9527-c1956e65cf45 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:25.177742 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-e64d4cbc-fc35-46f2-9527-c1956e65cf45 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a" status: 200 len: 1366 microversion: 2.1 time: 0.161639 Oct 30 01:01:25.178511 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 341/677] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:25 2022] GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a => generated 1366 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:25.793874 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-fd0359ad-088a-47d8-96cb-886a6de16682 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:25.794875 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-fd0359ad-088a-47d8-96cb-886a6de16682 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:25.804990 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-fd0359ad-088a-47d8-96cb-886a6de16682 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:25.805829 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-fd0359ad-088a-47d8-96cb-886a6de16682 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:25.806564 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-fd0359ad-088a-47d8-96cb-886a6de16682 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:25.807686 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-fd0359ad-088a-47d8-96cb-886a6de16682 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:25.808473 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-fd0359ad-088a-47d8-96cb-886a6de16682 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:25.809170 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-fd0359ad-088a-47d8-96cb-886a6de16682 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:25.946959 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-fd0359ad-088a-47d8-96cb-886a6de16682 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:25.956012 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-fd0359ad-088a-47d8-96cb-886a6de16682 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:25.957450 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-fd0359ad-088a-47d8-96cb-886a6de16682 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:25.958620 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-fd0359ad-088a-47d8-96cb-886a6de16682 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1810 microversion: 2.20 time: 0.171715 Oct 30 01:01:25.959183 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 337/678] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:01:25 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1810 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:01:26.114843 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-e71b4847-9657-4afb-8fcb-721922f17ce8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:26.115866 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-e71b4847-9657-4afb-8fcb-721922f17ce8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:26.126802 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e71b4847-9657-4afb-8fcb-721922f17ce8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:26.127788 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e71b4847-9657-4afb-8fcb-721922f17ce8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:26.128300 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e71b4847-9657-4afb-8fcb-721922f17ce8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:26.128993 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e71b4847-9657-4afb-8fcb-721922f17ce8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:26.129518 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e71b4847-9657-4afb-8fcb-721922f17ce8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:26.129950 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e71b4847-9657-4afb-8fcb-721922f17ce8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:26.200469 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-8fd68f46-660a-453a-bd05-77beb241e7b8 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:26.201404 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-8fd68f46-660a-453a-bd05-77beb241e7b8 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5632075c-7062-434e-b00c-bbf04c5ac04a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:26.208090 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8fd68f46-660a-453a-bd05-77beb241e7b8 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:26.208567 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8fd68f46-660a-453a-bd05-77beb241e7b8 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:26.208993 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8fd68f46-660a-453a-bd05-77beb241e7b8 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:26.209700 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8fd68f46-660a-453a-bd05-77beb241e7b8 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:26.210115 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8fd68f46-660a-453a-bd05-77beb241e7b8 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:26.210620 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8fd68f46-660a-453a-bd05-77beb241e7b8 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:26.309708 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-e71b4847-9657-4afb-8fcb-721922f17ce8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:26.319640 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-e71b4847-9657-4afb-8fcb-721922f17ce8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1371 microversion: 2.1 time: 0.211654 Oct 30 01:01:26.320509 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 342/679] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:26 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1371 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:26.354203 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-8fd68f46-660a-453a-bd05-77beb241e7b8 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:26.363576 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-8fd68f46-660a-453a-bd05-77beb241e7b8 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a" status: 200 len: 1366 microversion: 2.1 time: 0.167009 Oct 30 01:01:26.364173 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 338/680] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:26 2022] GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a => generated 1366 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:26.988798 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-40fa1eec-08c0-473d-8c10-71e28d5357cb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:26.989540 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-40fa1eec-08c0-473d-8c10-71e28d5357cb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:26.996906 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-40fa1eec-08c0-473d-8c10-71e28d5357cb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:26.997479 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-40fa1eec-08c0-473d-8c10-71e28d5357cb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:26.997991 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-40fa1eec-08c0-473d-8c10-71e28d5357cb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:26.998797 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-40fa1eec-08c0-473d-8c10-71e28d5357cb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:26.999328 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-40fa1eec-08c0-473d-8c10-71e28d5357cb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:26.999848 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-40fa1eec-08c0-473d-8c10-71e28d5357cb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:27.143782 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-40fa1eec-08c0-473d-8c10-71e28d5357cb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:27.153057 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-40fa1eec-08c0-473d-8c10-71e28d5357cb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:27.154691 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-40fa1eec-08c0-473d-8c10-71e28d5357cb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:27.155966 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-40fa1eec-08c0-473d-8c10-71e28d5357cb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1810 microversion: 2.20 time: 0.173494 Oct 30 01:01:27.156540 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 343/681] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:01:26 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1810 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:01:27.347455 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-b80f8ae5-cd36-42cb-923a-bc7b0f064d26 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:27.348025 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-b80f8ae5-cd36-42cb-923a-bc7b0f064d26 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:27.355974 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b80f8ae5-cd36-42cb-923a-bc7b0f064d26 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:27.356463 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b80f8ae5-cd36-42cb-923a-bc7b0f064d26 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:27.356892 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b80f8ae5-cd36-42cb-923a-bc7b0f064d26 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:27.357592 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b80f8ae5-cd36-42cb-923a-bc7b0f064d26 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:27.358039 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b80f8ae5-cd36-42cb-923a-bc7b0f064d26 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:27.358459 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b80f8ae5-cd36-42cb-923a-bc7b0f064d26 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:27.386584 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-a4eaa6ad-5c1f-42a6-9fc8-7e327328100c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:27.387198 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-a4eaa6ad-5c1f-42a6-9fc8-7e327328100c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5632075c-7062-434e-b00c-bbf04c5ac04a] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:27.394280 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a4eaa6ad-5c1f-42a6-9fc8-7e327328100c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:27.394903 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a4eaa6ad-5c1f-42a6-9fc8-7e327328100c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:27.395888 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a4eaa6ad-5c1f-42a6-9fc8-7e327328100c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:27.396731 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a4eaa6ad-5c1f-42a6-9fc8-7e327328100c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:27.397250 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a4eaa6ad-5c1f-42a6-9fc8-7e327328100c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:27.397757 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a4eaa6ad-5c1f-42a6-9fc8-7e327328100c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:27.540215 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-b80f8ae5-cd36-42cb-923a-bc7b0f064d26 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:27.548625 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-b80f8ae5-cd36-42cb-923a-bc7b0f064d26 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1371 microversion: 2.1 time: 0.207976 Oct 30 01:01:27.549787 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 339/682] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:27 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1371 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:27.571890 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-a4eaa6ad-5c1f-42a6-9fc8-7e327328100c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:27.580204 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-a4eaa6ad-5c1f-42a6-9fc8-7e327328100c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a" status: 200 len: 1366 microversion: 2.1 time: 0.197470 Oct 30 01:01:27.580772 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 344/683] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:27 2022] GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a => generated 1366 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 01:01:28.189220 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-d1b06bfd-b910-4164-9f8b-7fde1b75c113 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:28.189991 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-d1b06bfd-b910-4164-9f8b-7fde1b75c113 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:28.196896 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d1b06bfd-b910-4164-9f8b-7fde1b75c113 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:28.197378 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d1b06bfd-b910-4164-9f8b-7fde1b75c113 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:28.197808 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d1b06bfd-b910-4164-9f8b-7fde1b75c113 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:28.198504 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d1b06bfd-b910-4164-9f8b-7fde1b75c113 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:28.198955 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d1b06bfd-b910-4164-9f8b-7fde1b75c113 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:28.199403 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d1b06bfd-b910-4164-9f8b-7fde1b75c113 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:28.337750 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-d1b06bfd-b910-4164-9f8b-7fde1b75c113 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:28.346594 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-d1b06bfd-b910-4164-9f8b-7fde1b75c113 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:28.348141 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-d1b06bfd-b910-4164-9f8b-7fde1b75c113 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:28.349317 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-d1b06bfd-b910-4164-9f8b-7fde1b75c113 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1810 microversion: 2.20 time: 0.164607 Oct 30 01:01:28.349879 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 340/684] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:01:28 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1810 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:01:28.571353 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-7c488d4a-29c3-4fbc-ae92-f302ddf000ff tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:28.571979 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-7c488d4a-29c3-4fbc-ae92-f302ddf000ff tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:28.578667 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7c488d4a-29c3-4fbc-ae92-f302ddf000ff tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:28.579153 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7c488d4a-29c3-4fbc-ae92-f302ddf000ff tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:28.579578 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7c488d4a-29c3-4fbc-ae92-f302ddf000ff tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:28.580288 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7c488d4a-29c3-4fbc-ae92-f302ddf000ff tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:28.580729 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7c488d4a-29c3-4fbc-ae92-f302ddf000ff tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:28.581144 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7c488d4a-29c3-4fbc-ae92-f302ddf000ff tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:28.603540 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-b02cfb55-f3bd-4281-a94a-811f3d49f921 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:28.604182 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-b02cfb55-f3bd-4281-a94a-811f3d49f921 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5632075c-7062-434e-b00c-bbf04c5ac04a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:28.613592 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b02cfb55-f3bd-4281-a94a-811f3d49f921 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:28.614177 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b02cfb55-f3bd-4281-a94a-811f3d49f921 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:28.614689 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b02cfb55-f3bd-4281-a94a-811f3d49f921 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:28.615506 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b02cfb55-f3bd-4281-a94a-811f3d49f921 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:28.616050 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b02cfb55-f3bd-4281-a94a-811f3d49f921 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:28.616535 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b02cfb55-f3bd-4281-a94a-811f3d49f921 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:28.810235 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-b02cfb55-f3bd-4281-a94a-811f3d49f921 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:28.813616 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-7c488d4a-29c3-4fbc-ae92-f302ddf000ff tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:28.824553 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-b02cfb55-f3bd-4281-a94a-811f3d49f921 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a" status: 200 len: 1366 microversion: 2.1 time: 0.225259 Oct 30 01:01:28.825128 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 341/685] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:28 2022] GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a => generated 1366 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:28.836794 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-7c488d4a-29c3-4fbc-ae92-f302ddf000ff tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1371 microversion: 2.1 time: 0.268820 Oct 30 01:01:28.837696 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 345/686] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:28 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1371 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:29.376253 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-b8d30990-f62d-4aa1-a43b-62480eede5df tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:29.376868 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-b8d30990-f62d-4aa1-a43b-62480eede5df tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:29.383279 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b8d30990-f62d-4aa1-a43b-62480eede5df tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:29.383760 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b8d30990-f62d-4aa1-a43b-62480eede5df tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:29.384230 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b8d30990-f62d-4aa1-a43b-62480eede5df tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:29.384924 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b8d30990-f62d-4aa1-a43b-62480eede5df tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:29.385371 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b8d30990-f62d-4aa1-a43b-62480eede5df tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:29.385793 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b8d30990-f62d-4aa1-a43b-62480eede5df tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:29.517238 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-b8d30990-f62d-4aa1-a43b-62480eede5df tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:29.526221 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-b8d30990-f62d-4aa1-a43b-62480eede5df tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:29.527659 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-b8d30990-f62d-4aa1-a43b-62480eede5df tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:29.528847 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-b8d30990-f62d-4aa1-a43b-62480eede5df tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1750 microversion: 2.20 time: 0.156774 Oct 30 01:01:29.529406 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 342/687] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:01:29 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1750 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:01:29.854830 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-7a44222a-486b-40a7-ba17-59ecc9cb6fe7 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:29.855521 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-7a44222a-486b-40a7-ba17-59ecc9cb6fe7 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5632075c-7062-434e-b00c-bbf04c5ac04a] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:29.859905 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-24ff4661-8198-4d7e-a797-f8e01dcf48d4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:29.860578 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-24ff4661-8198-4d7e-a797-f8e01dcf48d4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:29.862298 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7a44222a-486b-40a7-ba17-59ecc9cb6fe7 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:29.862768 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7a44222a-486b-40a7-ba17-59ecc9cb6fe7 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:29.863405 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7a44222a-486b-40a7-ba17-59ecc9cb6fe7 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:29.864143 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7a44222a-486b-40a7-ba17-59ecc9cb6fe7 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:29.864826 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7a44222a-486b-40a7-ba17-59ecc9cb6fe7 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:29.865219 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7a44222a-486b-40a7-ba17-59ecc9cb6fe7 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:29.868465 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-24ff4661-8198-4d7e-a797-f8e01dcf48d4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:29.868973 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-24ff4661-8198-4d7e-a797-f8e01dcf48d4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:29.869860 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-24ff4661-8198-4d7e-a797-f8e01dcf48d4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:29.870572 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-24ff4661-8198-4d7e-a797-f8e01dcf48d4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:29.870994 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-24ff4661-8198-4d7e-a797-f8e01dcf48d4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:29.871434 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-24ff4661-8198-4d7e-a797-f8e01dcf48d4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:29.888590 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell1:SELECT=100 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:01:30.288867 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-7a44222a-486b-40a7-ba17-59ecc9cb6fe7 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:30.292104 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-24ff4661-8198-4d7e-a797-f8e01dcf48d4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:30.297171 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-7a44222a-486b-40a7-ba17-59ecc9cb6fe7 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a" status: 200 len: 1366 microversion: 2.1 time: 0.446551 Oct 30 01:01:30.297777 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 346/688] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:29 2022] GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a => generated 1366 bytes in 448 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:30.300049 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-24ff4661-8198-4d7e-a797-f8e01dcf48d4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1371 microversion: 2.1 time: 0.444896 Oct 30 01:01:30.300665 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 343/689] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:29 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1371 bytes in 446 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:30.559123 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-2d32c190-c612-4a14-a7f0-07d4f2d5e11c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:30.560192 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-2d32c190-c612-4a14-a7f0-07d4f2d5e11c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:30.568938 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2d32c190-c612-4a14-a7f0-07d4f2d5e11c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:30.569419 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2d32c190-c612-4a14-a7f0-07d4f2d5e11c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:30.569848 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2d32c190-c612-4a14-a7f0-07d4f2d5e11c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:30.570532 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2d32c190-c612-4a14-a7f0-07d4f2d5e11c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:30.570982 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2d32c190-c612-4a14-a7f0-07d4f2d5e11c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:30.571399 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2d32c190-c612-4a14-a7f0-07d4f2d5e11c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:30.706760 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-2d32c190-c612-4a14-a7f0-07d4f2d5e11c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:30.715754 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-2d32c190-c612-4a14-a7f0-07d4f2d5e11c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:30.717221 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-2d32c190-c612-4a14-a7f0-07d4f2d5e11c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:30.718399 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-2d32c190-c612-4a14-a7f0-07d4f2d5e11c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1752 microversion: 2.20 time: 0.166000 Oct 30 01:01:30.718975 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 347/690] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:01:30 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1752 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:01:31.324377 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-3b9a9561-db13-4378-8a1e-23c55e043c9e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:31.325378 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-3b9a9561-db13-4378-8a1e-23c55e043c9e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5632075c-7062-434e-b00c-bbf04c5ac04a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:31.329210 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-becfdc7c-35a7-475f-af2f-023fe180172d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:31.330305 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-becfdc7c-35a7-475f-af2f-023fe180172d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:31.336793 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3b9a9561-db13-4378-8a1e-23c55e043c9e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:31.337646 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3b9a9561-db13-4378-8a1e-23c55e043c9e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:31.338330 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3b9a9561-db13-4378-8a1e-23c55e043c9e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:31.339464 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3b9a9561-db13-4378-8a1e-23c55e043c9e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:31.340232 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3b9a9561-db13-4378-8a1e-23c55e043c9e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:31.340923 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3b9a9561-db13-4378-8a1e-23c55e043c9e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:31.341962 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-becfdc7c-35a7-475f-af2f-023fe180172d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:31.342794 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-becfdc7c-35a7-475f-af2f-023fe180172d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:31.343854 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-becfdc7c-35a7-475f-af2f-023fe180172d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:31.345316 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-becfdc7c-35a7-475f-af2f-023fe180172d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:31.346109 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-becfdc7c-35a7-475f-af2f-023fe180172d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:31.346877 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-becfdc7c-35a7-475f-af2f-023fe180172d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:31.537442 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-3b9a9561-db13-4378-8a1e-23c55e043c9e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:31.540346 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-becfdc7c-35a7-475f-af2f-023fe180172d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:31.545700 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-3b9a9561-db13-4378-8a1e-23c55e043c9e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a" status: 200 len: 1366 microversion: 2.1 time: 0.227864 Oct 30 01:01:31.546280 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 344/691] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:31 2022] GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a => generated 1366 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:31.549091 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-becfdc7c-35a7-475f-af2f-023fe180172d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1371 microversion: 2.1 time: 0.226969 Oct 30 01:01:31.549663 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 348/692] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:31 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1371 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:31.744203 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-6320b38e-9113-4400-b9c4-f4e3dc4a372c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:31.744837 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-6320b38e-9113-4400-b9c4-f4e3dc4a372c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:31.751875 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6320b38e-9113-4400-b9c4-f4e3dc4a372c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:31.752498 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6320b38e-9113-4400-b9c4-f4e3dc4a372c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:31.752953 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6320b38e-9113-4400-b9c4-f4e3dc4a372c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:31.753992 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6320b38e-9113-4400-b9c4-f4e3dc4a372c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:31.754704 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6320b38e-9113-4400-b9c4-f4e3dc4a372c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:31.755111 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6320b38e-9113-4400-b9c4-f4e3dc4a372c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:31.896048 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-6320b38e-9113-4400-b9c4-f4e3dc4a372c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:31.905073 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-6320b38e-9113-4400-b9c4-f4e3dc4a372c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:31.906486 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-6320b38e-9113-4400-b9c4-f4e3dc4a372c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:31.907633 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-6320b38e-9113-4400-b9c4-f4e3dc4a372c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1752 microversion: 2.20 time: 0.167481 Oct 30 01:01:31.908217 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 345/693] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:01:31 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1752 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:01:32.571445 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-4e5b4093-4d8d-4d03-bae2-8802414ac546 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:32.572186 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-4e5b4093-4d8d-4d03-bae2-8802414ac546 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5632075c-7062-434e-b00c-bbf04c5ac04a] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:32.573557 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-f8f4e3cf-597b-48c9-9d04-eaf99046d721 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:32.574242 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-f8f4e3cf-597b-48c9-9d04-eaf99046d721 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:32.581019 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4e5b4093-4d8d-4d03-bae2-8802414ac546 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:32.581487 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f8f4e3cf-597b-48c9-9d04-eaf99046d721 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:32.581661 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4e5b4093-4d8d-4d03-bae2-8802414ac546 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:32.582081 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f8f4e3cf-597b-48c9-9d04-eaf99046d721 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:32.582237 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4e5b4093-4d8d-4d03-bae2-8802414ac546 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:32.582560 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f8f4e3cf-597b-48c9-9d04-eaf99046d721 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:32.582822 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4e5b4093-4d8d-4d03-bae2-8802414ac546 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:32.583312 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f8f4e3cf-597b-48c9-9d04-eaf99046d721 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:32.583444 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4e5b4093-4d8d-4d03-bae2-8802414ac546 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:32.583854 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f8f4e3cf-597b-48c9-9d04-eaf99046d721 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:32.583999 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4e5b4093-4d8d-4d03-bae2-8802414ac546 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:32.584321 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f8f4e3cf-597b-48c9-9d04-eaf99046d721 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:32.776060 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-f8f4e3cf-597b-48c9-9d04-eaf99046d721 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:32.779482 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-4e5b4093-4d8d-4d03-bae2-8802414ac546 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:32.783946 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-f8f4e3cf-597b-48c9-9d04-eaf99046d721 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1371 microversion: 2.1 time: 0.215004 Oct 30 01:01:32.784520 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 346/694] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:32 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1371 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:32.787357 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-4e5b4093-4d8d-4d03-bae2-8802414ac546 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a" status: 200 len: 1366 microversion: 2.1 time: 0.220646 Oct 30 01:01:32.787946 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 349/695] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:32 2022] GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a => generated 1366 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:32.936168 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-c6abeaa5-8070-4f71-ac5f-8171226ca7ec tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:32.937168 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-c6abeaa5-8070-4f71-ac5f-8171226ca7ec tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:32.947570 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c6abeaa5-8070-4f71-ac5f-8171226ca7ec tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:32.948407 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c6abeaa5-8070-4f71-ac5f-8171226ca7ec tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:32.949113 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c6abeaa5-8070-4f71-ac5f-8171226ca7ec tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:32.950207 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c6abeaa5-8070-4f71-ac5f-8171226ca7ec tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:32.951007 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c6abeaa5-8070-4f71-ac5f-8171226ca7ec tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:32.951641 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c6abeaa5-8070-4f71-ac5f-8171226ca7ec tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:33.086231 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-c6abeaa5-8070-4f71-ac5f-8171226ca7ec tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:33.095004 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-c6abeaa5-8070-4f71-ac5f-8171226ca7ec tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:33.096545 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-c6abeaa5-8070-4f71-ac5f-8171226ca7ec tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:33.097887 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-c6abeaa5-8070-4f71-ac5f-8171226ca7ec tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1752 microversion: 2.20 time: 0.168178 Oct 30 01:01:33.098383 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 347/696] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:01:32 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1752 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:01:33.807358 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-ff72034f-a2a4-47bf-9ab7-06cfeb787e14 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:33.808002 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-ff72034f-a2a4-47bf-9ab7-06cfeb787e14 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:33.809850 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-40e974be-a3d0-4520-831f-debc7a91cdbe tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:33.810454 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-40e974be-a3d0-4520-831f-debc7a91cdbe tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5632075c-7062-434e-b00c-bbf04c5ac04a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:33.814492 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ff72034f-a2a4-47bf-9ab7-06cfeb787e14 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:33.814996 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ff72034f-a2a4-47bf-9ab7-06cfeb787e14 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:33.815415 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ff72034f-a2a4-47bf-9ab7-06cfeb787e14 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:33.816150 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ff72034f-a2a4-47bf-9ab7-06cfeb787e14 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:33.816600 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ff72034f-a2a4-47bf-9ab7-06cfeb787e14 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:33.817093 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ff72034f-a2a4-47bf-9ab7-06cfeb787e14 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:33.817457 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-40e974be-a3d0-4520-831f-debc7a91cdbe tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:33.817956 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-40e974be-a3d0-4520-831f-debc7a91cdbe tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:33.818382 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-40e974be-a3d0-4520-831f-debc7a91cdbe tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:33.819055 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-40e974be-a3d0-4520-831f-debc7a91cdbe tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:33.819501 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-40e974be-a3d0-4520-831f-debc7a91cdbe tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:33.819950 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-40e974be-a3d0-4520-831f-debc7a91cdbe tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:34.008417 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-ff72034f-a2a4-47bf-9ab7-06cfeb787e14 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:34.011513 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-40e974be-a3d0-4520-831f-debc7a91cdbe tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:34.017321 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-ff72034f-a2a4-47bf-9ab7-06cfeb787e14 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1371 microversion: 2.1 time: 0.214454 Oct 30 01:01:34.017961 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 350/697] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:33 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1371 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:34.019050 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-40e974be-a3d0-4520-831f-debc7a91cdbe tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a" status: 200 len: 1366 microversion: 2.1 time: 0.213019 Oct 30 01:01:34.019614 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 348/698] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:33 2022] GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a => generated 1366 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:34.125535 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-d22134c8-9184-4312-8f39-686a647eff1f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:34.125535 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-d22134c8-9184-4312-8f39-686a647eff1f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:34.130260 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d22134c8-9184-4312-8f39-686a647eff1f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:34.130820 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d22134c8-9184-4312-8f39-686a647eff1f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:34.131380 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d22134c8-9184-4312-8f39-686a647eff1f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:34.132168 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d22134c8-9184-4312-8f39-686a647eff1f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:34.132709 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d22134c8-9184-4312-8f39-686a647eff1f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:34.133201 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d22134c8-9184-4312-8f39-686a647eff1f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:34.268179 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-d22134c8-9184-4312-8f39-686a647eff1f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:34.282123 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-d22134c8-9184-4312-8f39-686a647eff1f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:34.284315 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-d22134c8-9184-4312-8f39-686a647eff1f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:34.285967 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-d22134c8-9184-4312-8f39-686a647eff1f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1752 microversion: 2.20 time: 0.167282 Oct 30 01:01:34.286568 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 351/699] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:01:34 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1752 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:01:35.041891 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-7fc41928-e013-4995-ab21-e56c9238162d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:35.042546 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-1e03478e-bf0b-4fc0-8f24-e476e6f61e50 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:35.042923 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-7fc41928-e013-4995-ab21-e56c9238162d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:35.043033 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-1e03478e-bf0b-4fc0-8f24-e476e6f61e50 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5632075c-7062-434e-b00c-bbf04c5ac04a] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:35.049449 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1e03478e-bf0b-4fc0-8f24-e476e6f61e50 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:35.049610 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7fc41928-e013-4995-ab21-e56c9238162d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:35.049968 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1e03478e-bf0b-4fc0-8f24-e476e6f61e50 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:35.050121 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7fc41928-e013-4995-ab21-e56c9238162d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:35.050394 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1e03478e-bf0b-4fc0-8f24-e476e6f61e50 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:35.050548 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7fc41928-e013-4995-ab21-e56c9238162d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:35.051114 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7fc41928-e013-4995-ab21-e56c9238162d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:35.051269 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1e03478e-bf0b-4fc0-8f24-e476e6f61e50 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:35.051632 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1e03478e-bf0b-4fc0-8f24-e476e6f61e50 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:35.051801 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7fc41928-e013-4995-ab21-e56c9238162d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:35.052078 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1e03478e-bf0b-4fc0-8f24-e476e6f61e50 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:35.052185 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7fc41928-e013-4995-ab21-e56c9238162d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:35.249895 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-1e03478e-bf0b-4fc0-8f24-e476e6f61e50 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:35.254205 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-7fc41928-e013-4995-ab21-e56c9238162d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:35.257719 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell1:SELECT=100 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:01:35.258484 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-1e03478e-bf0b-4fc0-8f24-e476e6f61e50 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a" status: 200 len: 1366 microversion: 2.1 time: 0.220445 Oct 30 01:01:35.259071 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 352/700] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:35 2022] GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a => generated 1366 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:35.264378 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-7fc41928-e013-4995-ab21-e56c9238162d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1371 microversion: 2.1 time: 0.225994 Oct 30 01:01:35.265235 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 349/701] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:35 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1371 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:35.310632 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-aa811a8a-2c55-4066-9552-71f10915ac7d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:35.311240 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-aa811a8a-2c55-4066-9552-71f10915ac7d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:35.317796 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-aa811a8a-2c55-4066-9552-71f10915ac7d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:35.318289 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-aa811a8a-2c55-4066-9552-71f10915ac7d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:35.318720 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-aa811a8a-2c55-4066-9552-71f10915ac7d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:35.319412 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-aa811a8a-2c55-4066-9552-71f10915ac7d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:35.319900 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-aa811a8a-2c55-4066-9552-71f10915ac7d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:35.320321 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-aa811a8a-2c55-4066-9552-71f10915ac7d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:35.454507 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-aa811a8a-2c55-4066-9552-71f10915ac7d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:35.463490 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-aa811a8a-2c55-4066-9552-71f10915ac7d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:35.465126 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-aa811a8a-2c55-4066-9552-71f10915ac7d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:35.466296 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-aa811a8a-2c55-4066-9552-71f10915ac7d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1752 microversion: 2.20 time: 0.159544 Oct 30 01:01:35.466860 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 353/702] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:01:35 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1752 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:01:36.285668 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-ce4eab77-e97b-487e-bd57-09e1f343f29b tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:36.286570 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-ce4eab77-e97b-487e-bd57-09e1f343f29b tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5632075c-7062-434e-b00c-bbf04c5ac04a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:36.291981 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-2ad1a3c4-100b-4765-b916-9ee8637c6b36 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:36.292601 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-2ad1a3c4-100b-4765-b916-9ee8637c6b36 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:36.293822 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ce4eab77-e97b-487e-bd57-09e1f343f29b tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:36.294282 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ce4eab77-e97b-487e-bd57-09e1f343f29b tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:36.294723 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ce4eab77-e97b-487e-bd57-09e1f343f29b tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:36.295422 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ce4eab77-e97b-487e-bd57-09e1f343f29b tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:36.295921 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ce4eab77-e97b-487e-bd57-09e1f343f29b tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:36.296336 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ce4eab77-e97b-487e-bd57-09e1f343f29b tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:36.299037 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2ad1a3c4-100b-4765-b916-9ee8637c6b36 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:36.299545 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2ad1a3c4-100b-4765-b916-9ee8637c6b36 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:36.300015 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2ad1a3c4-100b-4765-b916-9ee8637c6b36 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:36.300723 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2ad1a3c4-100b-4765-b916-9ee8637c6b36 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:36.301176 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2ad1a3c4-100b-4765-b916-9ee8637c6b36 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:36.301616 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2ad1a3c4-100b-4765-b916-9ee8637c6b36 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:36.492235 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-ce4eab77-e97b-487e-bd57-09e1f343f29b tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:36.495520 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-2ad1a3c4-100b-4765-b916-9ee8637c6b36 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:36.500894 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-ce4eab77-e97b-487e-bd57-09e1f343f29b tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a" status: 200 len: 1366 microversion: 2.1 time: 0.218967 Oct 30 01:01:36.501604 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 350/703] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:36 2022] GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a => generated 1366 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:36.504840 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-2ad1a3c4-100b-4765-b916-9ee8637c6b36 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1371 microversion: 2.1 time: 0.216600 Oct 30 01:01:36.505465 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 354/704] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:36 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1371 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:36.508964 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-069e8c56-445c-420c-ae24-be097468d3cb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:36.509803 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-069e8c56-445c-420c-ae24-be097468d3cb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:36.516230 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-069e8c56-445c-420c-ae24-be097468d3cb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:36.516820 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-069e8c56-445c-420c-ae24-be097468d3cb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:36.517345 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-069e8c56-445c-420c-ae24-be097468d3cb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:36.518032 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-069e8c56-445c-420c-ae24-be097468d3cb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:36.518466 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-069e8c56-445c-420c-ae24-be097468d3cb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:36.518876 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-069e8c56-445c-420c-ae24-be097468d3cb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:36.646350 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-069e8c56-445c-420c-ae24-be097468d3cb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:36.655646 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-069e8c56-445c-420c-ae24-be097468d3cb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:36.657047 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-069e8c56-445c-420c-ae24-be097468d3cb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:36.658200 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-069e8c56-445c-420c-ae24-be097468d3cb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1752 microversion: 2.20 time: 0.153474 Oct 30 01:01:36.658761 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 351/705] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:01:36 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1752 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:01:37.527946 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-9b04d51d-dec7-4da4-8f66-31c26b510353 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:37.528633 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-9b04d51d-dec7-4da4-8f66-31c26b510353 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5632075c-7062-434e-b00c-bbf04c5ac04a] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:37.530318 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-970706ea-c734-42d3-b858-98a9fa44a106 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:37.531120 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-970706ea-c734-42d3-b858-98a9fa44a106 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:37.539202 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9b04d51d-dec7-4da4-8f66-31c26b510353 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:37.539895 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-970706ea-c734-42d3-b858-98a9fa44a106 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:37.540054 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9b04d51d-dec7-4da4-8f66-31c26b510353 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:37.540497 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9b04d51d-dec7-4da4-8f66-31c26b510353 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:37.540670 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-970706ea-c734-42d3-b858-98a9fa44a106 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:37.541182 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-970706ea-c734-42d3-b858-98a9fa44a106 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:37.541416 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9b04d51d-dec7-4da4-8f66-31c26b510353 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:37.542008 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9b04d51d-dec7-4da4-8f66-31c26b510353 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:37.542140 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-970706ea-c734-42d3-b858-98a9fa44a106 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:37.542562 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9b04d51d-dec7-4da4-8f66-31c26b510353 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:37.542702 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-970706ea-c734-42d3-b858-98a9fa44a106 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:37.543211 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-970706ea-c734-42d3-b858-98a9fa44a106 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:37.737016 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-9b04d51d-dec7-4da4-8f66-31c26b510353 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:37.740482 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-970706ea-c734-42d3-b858-98a9fa44a106 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:37.744989 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-9b04d51d-dec7-4da4-8f66-31c26b510353 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a" status: 200 len: 1366 microversion: 2.1 time: 0.222410 Oct 30 01:01:37.745835 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 355/706] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:37 2022] GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a => generated 1366 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:37.748376 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-970706ea-c734-42d3-b858-98a9fa44a106 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1371 microversion: 2.1 time: 0.223439 Oct 30 01:01:37.750565 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-5433250c-b822-413c-9550-b51a3e4a9aff tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:37.751432 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-5433250c-b822-413c-9550-b51a3e4a9aff tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:37.755236 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 352/707] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:37 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1371 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:37.758563 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5433250c-b822-413c-9550-b51a3e4a9aff tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:37.759057 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5433250c-b822-413c-9550-b51a3e4a9aff tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:37.759492 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5433250c-b822-413c-9550-b51a3e4a9aff tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:37.760312 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5433250c-b822-413c-9550-b51a3e4a9aff tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:37.760790 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5433250c-b822-413c-9550-b51a3e4a9aff tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:37.761449 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5433250c-b822-413c-9550-b51a3e4a9aff tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:37.902914 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-5433250c-b822-413c-9550-b51a3e4a9aff tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:37.912089 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-5433250c-b822-413c-9550-b51a3e4a9aff tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:37.913570 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-5433250c-b822-413c-9550-b51a3e4a9aff tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:37.914751 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-5433250c-b822-413c-9550-b51a3e4a9aff tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1752 microversion: 2.20 time: 0.167920 Oct 30 01:01:37.915311 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 356/708] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:01:37 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1752 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:01:38.063031 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [req-6b8351a7-47d8-41b2-a8f2-6c329fed9103 req-fe8463e7-a1ee-459e-8ca9-8bd2af296ac8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "de41579d-0173-4dba-962d-a28c85c505de", "name": "network-vif-plugged", "status": "completed", "tag": "c4782435-5ac9-430e-95c9-c061a2747b76"}]} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:01:38.073437 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-6b8351a7-47d8-41b2-a8f2-6c329fed9103 req-fe8463e7-a1ee-459e-8ca9-8bd2af296ac8 service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:38.073935 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-6b8351a7-47d8-41b2-a8f2-6c329fed9103 req-fe8463e7-a1ee-459e-8ca9-8bd2af296ac8 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:38.074348 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-6b8351a7-47d8-41b2-a8f2-6c329fed9103 req-fe8463e7-a1ee-459e-8ca9-8bd2af296ac8 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:38.086141 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.compute.server_external_events [req-6b8351a7-47d8-41b2-a8f2-6c329fed9103 req-fe8463e7-a1ee-459e-8ca9-8bd2af296ac8 service nova] Creating event network-vif-plugged:c4782435-5ac9-430e-95c9-c061a2747b76 for instance de41579d-0173-4dba-962d-a28c85c505de on hv-cinder-maste Oct 30 01:01:38.091280 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [req-6b8351a7-47d8-41b2-a8f2-6c329fed9103 req-fe8463e7-a1ee-459e-8ca9-8bd2af296ac8 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031957 Oct 30 01:01:38.091956 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 353/709] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:01:38 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:01:38.769214 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-584b84d3-297e-41f9-a457-6fb1821f47d3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:38.769888 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-584b84d3-297e-41f9-a457-6fb1821f47d3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5632075c-7062-434e-b00c-bbf04c5ac04a] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:38.774590 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-8b25e3f0-b2c3-409d-9402-a30a34b30cb6 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:38.775288 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-8b25e3f0-b2c3-409d-9402-a30a34b30cb6 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:38.776789 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-584b84d3-297e-41f9-a457-6fb1821f47d3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:38.777395 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-584b84d3-297e-41f9-a457-6fb1821f47d3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:38.777705 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-584b84d3-297e-41f9-a457-6fb1821f47d3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:38.778417 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-584b84d3-297e-41f9-a457-6fb1821f47d3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:38.778872 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-584b84d3-297e-41f9-a457-6fb1821f47d3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:38.779270 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-584b84d3-297e-41f9-a457-6fb1821f47d3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:38.781684 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8b25e3f0-b2c3-409d-9402-a30a34b30cb6 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:38.782165 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8b25e3f0-b2c3-409d-9402-a30a34b30cb6 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:38.782709 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8b25e3f0-b2c3-409d-9402-a30a34b30cb6 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:38.783605 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8b25e3f0-b2c3-409d-9402-a30a34b30cb6 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:38.784201 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8b25e3f0-b2c3-409d-9402-a30a34b30cb6 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:38.784623 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8b25e3f0-b2c3-409d-9402-a30a34b30cb6 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:38.982524 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-8b25e3f0-b2c3-409d-9402-a30a34b30cb6 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:38.985543 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-584b84d3-297e-41f9-a457-6fb1821f47d3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:38.990896 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-8b25e3f0-b2c3-409d-9402-a30a34b30cb6 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1371 microversion: 2.1 time: 0.224231 Oct 30 01:01:38.991473 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 354/710] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:38 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1371 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:38.994200 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-584b84d3-297e-41f9-a457-6fb1821f47d3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a" status: 200 len: 1366 microversion: 2.1 time: 0.228803 Oct 30 01:01:38.994846 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 357/711] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:38 2022] GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a => generated 1366 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:39.000451 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-60187340-9677-4585-8554-12ee8b04829a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:39.001128 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-60187340-9677-4585-8554-12ee8b04829a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:39.007398 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-60187340-9677-4585-8554-12ee8b04829a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:39.007975 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-60187340-9677-4585-8554-12ee8b04829a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:39.008408 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-60187340-9677-4585-8554-12ee8b04829a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:39.009097 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-60187340-9677-4585-8554-12ee8b04829a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:39.009543 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-60187340-9677-4585-8554-12ee8b04829a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:39.009990 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-60187340-9677-4585-8554-12ee8b04829a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:39.046174 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-874b8785-c583-4491-afed-c054625a6d69 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:39.046669 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-874b8785-c583-4491-afed-c054625a6d69 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: db9ac414-31a5-4570-aa3f-37db3dac2049] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:39.053216 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-874b8785-c583-4491-afed-c054625a6d69 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:39.053636 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-874b8785-c583-4491-afed-c054625a6d69 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:39.054069 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-874b8785-c583-4491-afed-c054625a6d69 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:39.054765 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-874b8785-c583-4491-afed-c054625a6d69 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:39.055252 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-874b8785-c583-4491-afed-c054625a6d69 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:39.055664 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-874b8785-c583-4491-afed-c054625a6d69 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:39.140255 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-60187340-9677-4585-8554-12ee8b04829a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:39.149274 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-60187340-9677-4585-8554-12ee8b04829a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:39.150683 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-60187340-9677-4585-8554-12ee8b04829a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:39.151870 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-60187340-9677-4585-8554-12ee8b04829a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1752 microversion: 2.20 time: 0.155665 Oct 30 01:01:39.152443 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 355/712] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:01:38 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1752 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:01:39.184746 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.objects.instance [None req-874b8785-c583-4491-afed-c054625a6d69 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lazy-loading 'services' on Instance uuid db9ac414-31a5-4570-aa3f-37db3dac2049 {{(pid=118961) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 30 01:01:39.308159 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-874b8785-c583-4491-afed-c054625a6d69 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "DELETE /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049/os-volume_attachments/2b91b9a3-f9a3-43fd-a1ba-3be33aa06d1e" status: 202 len: 0 microversion: 2.1 time: 0.265649 Oct 30 01:01:39.308812 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 358/713] 10.222.0.16 () {58 vars in 1416 bytes} [Sun Oct 30 01:01:39 2022] DELETE /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049/os-volume_attachments/2b91b9a3-f9a3-43fd-a1ba-3be33aa06d1e => generated 0 bytes in 267 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 30 01:01:40.013819 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-1de0320a-8d51-4681-84ea-0f960fc2c61c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:40.014325 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-1de0320a-8d51-4681-84ea-0f960fc2c61c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:40.019395 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-fab397bf-0a36-4d53-a9fa-520e73a5945d tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:40.020641 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-fab397bf-0a36-4d53-a9fa-520e73a5945d tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5632075c-7062-434e-b00c-bbf04c5ac04a] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:40.020846 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1de0320a-8d51-4681-84ea-0f960fc2c61c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:40.021348 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1de0320a-8d51-4681-84ea-0f960fc2c61c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:40.021777 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1de0320a-8d51-4681-84ea-0f960fc2c61c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:40.022468 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1de0320a-8d51-4681-84ea-0f960fc2c61c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:40.022909 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1de0320a-8d51-4681-84ea-0f960fc2c61c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:40.023671 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1de0320a-8d51-4681-84ea-0f960fc2c61c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:40.026991 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-fab397bf-0a36-4d53-a9fa-520e73a5945d tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:40.027510 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-fab397bf-0a36-4d53-a9fa-520e73a5945d tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:40.027974 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-fab397bf-0a36-4d53-a9fa-520e73a5945d tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:40.028637 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-fab397bf-0a36-4d53-a9fa-520e73a5945d tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:40.029243 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-fab397bf-0a36-4d53-a9fa-520e73a5945d tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:40.029844 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-fab397bf-0a36-4d53-a9fa-520e73a5945d tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:40.240965 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-1de0320a-8d51-4681-84ea-0f960fc2c61c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:40.244180 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-fab397bf-0a36-4d53-a9fa-520e73a5945d tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:40.248964 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-1de0320a-8d51-4681-84ea-0f960fc2c61c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1371 microversion: 2.1 time: 0.239166 Oct 30 01:01:40.249536 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 356/714] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:40 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1371 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:40.252735 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-fab397bf-0a36-4d53-a9fa-520e73a5945d tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a" status: 200 len: 1366 microversion: 2.1 time: 0.238025 Oct 30 01:01:40.253575 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 359/715] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:40 2022] GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a => generated 1366 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:40.253982 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-f27523c7-4b09-4f9c-b2f8-9ac26ec3243d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:40.254742 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-f27523c7-4b09-4f9c-b2f8-9ac26ec3243d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:40.260380 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f27523c7-4b09-4f9c-b2f8-9ac26ec3243d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:40.260848 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f27523c7-4b09-4f9c-b2f8-9ac26ec3243d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:40.261270 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f27523c7-4b09-4f9c-b2f8-9ac26ec3243d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:40.261947 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f27523c7-4b09-4f9c-b2f8-9ac26ec3243d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:40.262387 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f27523c7-4b09-4f9c-b2f8-9ac26ec3243d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:40.262795 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f27523c7-4b09-4f9c-b2f8-9ac26ec3243d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:40.399331 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-f27523c7-4b09-4f9c-b2f8-9ac26ec3243d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:40.407745 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-f27523c7-4b09-4f9c-b2f8-9ac26ec3243d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:40.409195 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-f27523c7-4b09-4f9c-b2f8-9ac26ec3243d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:40.410350 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-f27523c7-4b09-4f9c-b2f8-9ac26ec3243d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1752 microversion: 2.20 time: 0.159914 Oct 30 01:01:40.410911 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 357/716] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:01:40 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1752 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:01:41.274536 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-355ae223-bccd-4d1e-85ba-f3d2194dc8f2 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:41.275115 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-355ae223-bccd-4d1e-85ba-f3d2194dc8f2 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:41.282068 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-4f1f35b1-0d2d-4edf-ac70-a26ca652a489 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:41.282765 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-4f1f35b1-0d2d-4edf-ac70-a26ca652a489 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5632075c-7062-434e-b00c-bbf04c5ac04a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:41.284330 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-355ae223-bccd-4d1e-85ba-f3d2194dc8f2 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:41.285074 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-355ae223-bccd-4d1e-85ba-f3d2194dc8f2 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:41.285525 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-355ae223-bccd-4d1e-85ba-f3d2194dc8f2 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:41.286410 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-355ae223-bccd-4d1e-85ba-f3d2194dc8f2 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:41.287002 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-355ae223-bccd-4d1e-85ba-f3d2194dc8f2 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:41.287632 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-355ae223-bccd-4d1e-85ba-f3d2194dc8f2 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:41.292650 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4f1f35b1-0d2d-4edf-ac70-a26ca652a489 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:41.293319 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4f1f35b1-0d2d-4edf-ac70-a26ca652a489 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:41.293769 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4f1f35b1-0d2d-4edf-ac70-a26ca652a489 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:41.294586 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4f1f35b1-0d2d-4edf-ac70-a26ca652a489 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:41.295081 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4f1f35b1-0d2d-4edf-ac70-a26ca652a489 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:41.295683 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4f1f35b1-0d2d-4edf-ac70-a26ca652a489 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:41.488017 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-355ae223-bccd-4d1e-85ba-f3d2194dc8f2 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:41.491920 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-4f1f35b1-0d2d-4edf-ac70-a26ca652a489 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:41.496316 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-355ae223-bccd-4d1e-85ba-f3d2194dc8f2 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1371 microversion: 2.1 time: 0.227187 Oct 30 01:01:41.507997 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 360/717] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:41 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1371 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:41.507997 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-5850c6e8-8669-4cd4-b68e-1676cdcce6f1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:41.508728 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-4f1f35b1-0d2d-4edf-ac70-a26ca652a489 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a" status: 200 len: 1366 microversion: 2.1 time: 0.225361 Oct 30 01:01:41.508825 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-5850c6e8-8669-4cd4-b68e-1676cdcce6f1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:41.509024 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5850c6e8-8669-4cd4-b68e-1676cdcce6f1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:41.509615 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5850c6e8-8669-4cd4-b68e-1676cdcce6f1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:41.510052 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5850c6e8-8669-4cd4-b68e-1676cdcce6f1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:41.510831 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5850c6e8-8669-4cd4-b68e-1676cdcce6f1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:41.511400 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5850c6e8-8669-4cd4-b68e-1676cdcce6f1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:41.511881 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5850c6e8-8669-4cd4-b68e-1676cdcce6f1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:41.515904 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 358/718] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:41 2022] GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a => generated 1366 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:41.650100 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-5850c6e8-8669-4cd4-b68e-1676cdcce6f1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:41.662220 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-5850c6e8-8669-4cd4-b68e-1676cdcce6f1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:41.663630 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-5850c6e8-8669-4cd4-b68e-1676cdcce6f1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:41.664808 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-5850c6e8-8669-4cd4-b68e-1676cdcce6f1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1752 microversion: 2.20 time: 0.166874 Oct 30 01:01:41.665369 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 361/719] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:01:41 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1752 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:01:42.521681 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-565d1957-c347-4d5c-b3b1-605369b20b53 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:42.522287 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-565d1957-c347-4d5c-b3b1-605369b20b53 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:42.525628 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-90679011-3a7c-4bf4-acba-b0913d3538a7 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:42.526565 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-90679011-3a7c-4bf4-acba-b0913d3538a7 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5632075c-7062-434e-b00c-bbf04c5ac04a] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:42.529485 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-565d1957-c347-4d5c-b3b1-605369b20b53 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:42.529945 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-565d1957-c347-4d5c-b3b1-605369b20b53 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:42.530395 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-565d1957-c347-4d5c-b3b1-605369b20b53 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:42.531106 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-565d1957-c347-4d5c-b3b1-605369b20b53 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:42.531581 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-565d1957-c347-4d5c-b3b1-605369b20b53 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:42.531976 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-565d1957-c347-4d5c-b3b1-605369b20b53 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:42.534218 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-90679011-3a7c-4bf4-acba-b0913d3538a7 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:42.534552 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-90679011-3a7c-4bf4-acba-b0913d3538a7 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:42.535209 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-90679011-3a7c-4bf4-acba-b0913d3538a7 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:42.535948 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-90679011-3a7c-4bf4-acba-b0913d3538a7 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:42.538139 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-90679011-3a7c-4bf4-acba-b0913d3538a7 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:42.538139 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-90679011-3a7c-4bf4-acba-b0913d3538a7 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:42.558540 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:01:42.790692 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-565d1957-c347-4d5c-b3b1-605369b20b53 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:42.800204 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-565d1957-c347-4d5c-b3b1-605369b20b53 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1565 microversion: 2.1 time: 0.282654 Oct 30 01:01:42.801337 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 359/720] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:42 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1565 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:42.806701 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [req-cc5d55c1-1e9d-473a-9495-ed8525a51484 req-5eeb427f-16e0-43e3-8754-81de9cb5b7f8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5632075c-7062-434e-b00c-bbf04c5ac04a", "name": "network-vif-plugged", "status": "completed", "tag": "c9fe8633-400b-4adf-99ff-00c04190ea67"}]} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:01:42.816611 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-cc5d55c1-1e9d-473a-9495-ed8525a51484 req-5eeb427f-16e0-43e3-8754-81de9cb5b7f8 service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:42.817332 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-cc5d55c1-1e9d-473a-9495-ed8525a51484 req-5eeb427f-16e0-43e3-8754-81de9cb5b7f8 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:42.818353 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-cc5d55c1-1e9d-473a-9495-ed8525a51484 req-5eeb427f-16e0-43e3-8754-81de9cb5b7f8 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:42.830680 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.compute.server_external_events [req-cc5d55c1-1e9d-473a-9495-ed8525a51484 req-5eeb427f-16e0-43e3-8754-81de9cb5b7f8 service nova] Creating event network-vif-plugged:c9fe8633-400b-4adf-99ff-00c04190ea67 for instance 5632075c-7062-434e-b00c-bbf04c5ac04a on hv-cinder-maste Oct 30 01:01:42.835750 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [req-cc5d55c1-1e9d-473a-9495-ed8525a51484 req-5eeb427f-16e0-43e3-8754-81de9cb5b7f8 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032545 Oct 30 01:01:42.836372 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 360/721] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:01:42 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:01:42.839018 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-90679011-3a7c-4bf4-acba-b0913d3538a7 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:42.843151 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-27280cdb-73ca-49ab-b262-328aec7e15ac tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:42.844059 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-27280cdb-73ca-49ab-b262-328aec7e15ac tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:42.847140 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-90679011-3a7c-4bf4-acba-b0913d3538a7 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a" status: 200 len: 1366 microversion: 2.1 time: 0.325825 Oct 30 01:01:42.847754 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 362/722] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:42 2022] GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a => generated 1366 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:42.851003 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-27280cdb-73ca-49ab-b262-328aec7e15ac tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:42.851520 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-27280cdb-73ca-49ab-b262-328aec7e15ac tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:42.852155 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-27280cdb-73ca-49ab-b262-328aec7e15ac tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:42.852875 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-b12bbdf0-ac0f-44d9-970b-0b53733ca1fd tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:42.853185 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-27280cdb-73ca-49ab-b262-328aec7e15ac tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:42.853536 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-b12bbdf0-ac0f-44d9-970b-0b53733ca1fd tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:42.853942 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-27280cdb-73ca-49ab-b262-328aec7e15ac tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:42.854498 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-27280cdb-73ca-49ab-b262-328aec7e15ac tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:42.862025 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b12bbdf0-ac0f-44d9-970b-0b53733ca1fd tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:42.862731 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b12bbdf0-ac0f-44d9-970b-0b53733ca1fd tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:42.863479 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b12bbdf0-ac0f-44d9-970b-0b53733ca1fd tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:42.864596 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b12bbdf0-ac0f-44d9-970b-0b53733ca1fd tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:42.865411 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b12bbdf0-ac0f-44d9-970b-0b53733ca1fd tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:42.866115 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b12bbdf0-ac0f-44d9-970b-0b53733ca1fd tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:42.986428 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-b12bbdf0-ac0f-44d9-970b-0b53733ca1fd tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.137432 Oct 30 01:01:42.987168 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 363/723] 10.222.0.16 () {58 vars in 1275 bytes} [Sun Oct 30 01:01:42 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de/os-interface => generated 284 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:01:43.038776 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-27280cdb-73ca-49ab-b262-328aec7e15ac tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:43.049120 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-27280cdb-73ca-49ab-b262-328aec7e15ac tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:43.050682 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-27280cdb-73ca-49ab-b262-328aec7e15ac tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:43.051932 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-27280cdb-73ca-49ab-b262-328aec7e15ac tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1752 microversion: 2.20 time: 0.214226 Oct 30 01:01:43.057635 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 361/724] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:01:42 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1752 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:01:43.872108 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-67dc342b-01a5-4822-a1f5-cda22b323b1c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:43.872674 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-67dc342b-01a5-4822-a1f5-cda22b323b1c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5632075c-7062-434e-b00c-bbf04c5ac04a] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:43.878972 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-67dc342b-01a5-4822-a1f5-cda22b323b1c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:43.879428 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-67dc342b-01a5-4822-a1f5-cda22b323b1c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:43.880039 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-67dc342b-01a5-4822-a1f5-cda22b323b1c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:43.880883 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-67dc342b-01a5-4822-a1f5-cda22b323b1c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:43.881438 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-67dc342b-01a5-4822-a1f5-cda22b323b1c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:43.881803 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-67dc342b-01a5-4822-a1f5-cda22b323b1c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:44.018464 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-67dc342b-01a5-4822-a1f5-cda22b323b1c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:44.026565 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-67dc342b-01a5-4822-a1f5-cda22b323b1c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a" status: 200 len: 1366 microversion: 2.1 time: 0.158916 Oct 30 01:01:44.027191 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 364/725] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:43 2022] GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a => generated 1366 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:44.077820 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-cfb5580d-9273-49e0-a399-bca756780118 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:44.078389 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-cfb5580d-9273-49e0-a399-bca756780118 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:44.085404 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-cfb5580d-9273-49e0-a399-bca756780118 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:44.085887 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-cfb5580d-9273-49e0-a399-bca756780118 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:44.086308 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-cfb5580d-9273-49e0-a399-bca756780118 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:44.087008 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-cfb5580d-9273-49e0-a399-bca756780118 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:44.087451 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-cfb5580d-9273-49e0-a399-bca756780118 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:44.087891 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-cfb5580d-9273-49e0-a399-bca756780118 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:44.227280 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-cfb5580d-9273-49e0-a399-bca756780118 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:44.236365 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-cfb5580d-9273-49e0-a399-bca756780118 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:44.237791 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-cfb5580d-9273-49e0-a399-bca756780118 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:44.238943 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-cfb5580d-9273-49e0-a399-bca756780118 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1808 microversion: 2.20 time: 0.165036 Oct 30 01:01:44.239465 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 362/726] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:01:44 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1808 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:01:44.361887 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [req-cb6013af-c5b8-4299-a9ac-2e2af1990b29 req-9fd1f83f-3d0d-45fc-8d4d-64bb62f387af service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "de41579d-0173-4dba-962d-a28c85c505de", "tag": "c4782435-5ac9-430e-95c9-c061a2747b76"}]} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:01:44.372076 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-cb6013af-c5b8-4299-a9ac-2e2af1990b29 req-9fd1f83f-3d0d-45fc-8d4d-64bb62f387af service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:44.372643 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-cb6013af-c5b8-4299-a9ac-2e2af1990b29 req-9fd1f83f-3d0d-45fc-8d4d-64bb62f387af service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:44.373523 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-cb6013af-c5b8-4299-a9ac-2e2af1990b29 req-9fd1f83f-3d0d-45fc-8d4d-64bb62f387af service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:44.388643 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.compute.server_external_events [req-cb6013af-c5b8-4299-a9ac-2e2af1990b29 req-9fd1f83f-3d0d-45fc-8d4d-64bb62f387af service nova] Creating event network-changed:c4782435-5ac9-430e-95c9-c061a2747b76 for instance de41579d-0173-4dba-962d-a28c85c505de on hv-cinder-maste Oct 30 01:01:44.395090 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [req-cb6013af-c5b8-4299-a9ac-2e2af1990b29 req-9fd1f83f-3d0d-45fc-8d4d-64bb62f387af service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.100574 Oct 30 01:01:44.396379 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 365/727] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:01:44 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:01:44.731934 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-a43dfe80-595c-4d93-a282-615c1fe96d37 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:44.732420 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-a43dfe80-595c-4d93-a282-615c1fe96d37 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: db9ac414-31a5-4570-aa3f-37db3dac2049] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:44.738370 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a43dfe80-595c-4d93-a282-615c1fe96d37 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:44.738841 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a43dfe80-595c-4d93-a282-615c1fe96d37 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:44.739263 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a43dfe80-595c-4d93-a282-615c1fe96d37 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:44.740015 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a43dfe80-595c-4d93-a282-615c1fe96d37 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:44.740438 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a43dfe80-595c-4d93-a282-615c1fe96d37 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:44.740845 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a43dfe80-595c-4d93-a282-615c1fe96d37 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:44.765562 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-a43dfe80-595c-4d93-a282-615c1fe96d37 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: db9ac414-31a5-4570-aa3f-37db3dac2049] Going to try to terminate instance {{(pid=118962) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 30 01:01:44.846381 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-a43dfe80-595c-4d93-a282-615c1fe96d37 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "DELETE /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049" status: 204 len: 0 microversion: 2.1 time: 0.117933 Oct 30 01:01:44.847258 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 363/728] 10.222.0.16 () {58 vars in 1239 bytes} [Sun Oct 30 01:01:44 2022] DELETE /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 30 01:01:44.856272 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-03575d8b-48db-411f-9950-e4d7be80ae44 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:44.857366 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-03575d8b-48db-411f-9950-e4d7be80ae44 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: db9ac414-31a5-4570-aa3f-37db3dac2049] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:44.863672 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-03575d8b-48db-411f-9950-e4d7be80ae44 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:44.864221 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-03575d8b-48db-411f-9950-e4d7be80ae44 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:44.864663 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-03575d8b-48db-411f-9950-e4d7be80ae44 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:44.865454 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-03575d8b-48db-411f-9950-e4d7be80ae44 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:44.866102 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-03575d8b-48db-411f-9950-e4d7be80ae44 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:44.866516 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-03575d8b-48db-411f-9950-e4d7be80ae44 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:44.999565 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-03575d8b-48db-411f-9950-e4d7be80ae44 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:45.009102 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-03575d8b-48db-411f-9950-e4d7be80ae44 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049" status: 200 len: 1561 microversion: 2.1 time: 0.157026 Oct 30 01:01:45.009704 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 366/729] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:44 2022] GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049 => generated 1561 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:45.051790 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-929ea688-0170-4016-a6e8-60ef7f1f9b1c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:45.052905 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-929ea688-0170-4016-a6e8-60ef7f1f9b1c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5632075c-7062-434e-b00c-bbf04c5ac04a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:45.059961 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-929ea688-0170-4016-a6e8-60ef7f1f9b1c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:45.060451 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-929ea688-0170-4016-a6e8-60ef7f1f9b1c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:45.060910 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-929ea688-0170-4016-a6e8-60ef7f1f9b1c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:45.061620 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-929ea688-0170-4016-a6e8-60ef7f1f9b1c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:45.062068 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-929ea688-0170-4016-a6e8-60ef7f1f9b1c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:45.062484 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-929ea688-0170-4016-a6e8-60ef7f1f9b1c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:45.229953 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-929ea688-0170-4016-a6e8-60ef7f1f9b1c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:45.240232 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-929ea688-0170-4016-a6e8-60ef7f1f9b1c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a" status: 200 len: 1366 microversion: 2.1 time: 0.193775 Oct 30 01:01:45.240877 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 364/730] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:45 2022] GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a => generated 1366 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:45.269703 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-d10934b4-00cc-4607-8190-c7a29be47e4a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:45.270515 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-d10934b4-00cc-4607-8190-c7a29be47e4a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:45.277761 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d10934b4-00cc-4607-8190-c7a29be47e4a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:45.278415 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d10934b4-00cc-4607-8190-c7a29be47e4a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:45.278944 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d10934b4-00cc-4607-8190-c7a29be47e4a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:45.279815 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d10934b4-00cc-4607-8190-c7a29be47e4a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:45.280478 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d10934b4-00cc-4607-8190-c7a29be47e4a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:45.280992 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d10934b4-00cc-4607-8190-c7a29be47e4a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:45.434017 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-d10934b4-00cc-4607-8190-c7a29be47e4a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:45.443119 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-d10934b4-00cc-4607-8190-c7a29be47e4a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:45.446563 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-d10934b4-00cc-4607-8190-c7a29be47e4a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:45.447972 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-d10934b4-00cc-4607-8190-c7a29be47e4a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1808 microversion: 2.20 time: 0.182923 Oct 30 01:01:45.448543 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 367/731] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:01:45 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1808 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:01:46.035482 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-862a1aa7-91f6-404b-9258-806721cd03c3 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:46.036158 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-862a1aa7-91f6-404b-9258-806721cd03c3 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: db9ac414-31a5-4570-aa3f-37db3dac2049] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:46.038534 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_api:SELECT=87,nova_api:UPDATE=1 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:01:46.043991 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-862a1aa7-91f6-404b-9258-806721cd03c3 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:46.044469 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-862a1aa7-91f6-404b-9258-806721cd03c3 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:46.044890 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-862a1aa7-91f6-404b-9258-806721cd03c3 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:46.045580 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-862a1aa7-91f6-404b-9258-806721cd03c3 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:46.046020 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-862a1aa7-91f6-404b-9258-806721cd03c3 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:46.046436 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-862a1aa7-91f6-404b-9258-806721cd03c3 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:46.183036 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-862a1aa7-91f6-404b-9258-806721cd03c3 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:46.191664 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-862a1aa7-91f6-404b-9258-806721cd03c3 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049" status: 200 len: 1515 microversion: 2.1 time: 0.161248 Oct 30 01:01:46.192272 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 365/732] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:46 2022] GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049 => generated 1515 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:46.268866 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-315e59eb-c9de-4a2e-a110-ab750669d195 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:46.269867 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-315e59eb-c9de-4a2e-a110-ab750669d195 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5632075c-7062-434e-b00c-bbf04c5ac04a] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:46.280723 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-315e59eb-c9de-4a2e-a110-ab750669d195 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:46.281519 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-315e59eb-c9de-4a2e-a110-ab750669d195 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:46.282276 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-315e59eb-c9de-4a2e-a110-ab750669d195 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:46.283374 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-315e59eb-c9de-4a2e-a110-ab750669d195 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:46.284122 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-315e59eb-c9de-4a2e-a110-ab750669d195 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:46.284858 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-315e59eb-c9de-4a2e-a110-ab750669d195 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:46.419692 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-315e59eb-c9de-4a2e-a110-ab750669d195 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:46.427953 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-315e59eb-c9de-4a2e-a110-ab750669d195 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a" status: 200 len: 1366 microversion: 2.1 time: 0.165414 Oct 30 01:01:46.428455 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 368/733] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:46 2022] GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a => generated 1366 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:46.473777 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-84f7eb3f-751c-4ad1-a44a-c4e78bb73de9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:46.474374 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-84f7eb3f-751c-4ad1-a44a-c4e78bb73de9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:46.481187 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-84f7eb3f-751c-4ad1-a44a-c4e78bb73de9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:46.481666 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-84f7eb3f-751c-4ad1-a44a-c4e78bb73de9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:46.482096 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-84f7eb3f-751c-4ad1-a44a-c4e78bb73de9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:46.482767 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-84f7eb3f-751c-4ad1-a44a-c4e78bb73de9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:46.483213 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-84f7eb3f-751c-4ad1-a44a-c4e78bb73de9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:46.483709 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-84f7eb3f-751c-4ad1-a44a-c4e78bb73de9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:46.487646 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:01:46.616496 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-84f7eb3f-751c-4ad1-a44a-c4e78bb73de9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:46.625331 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-84f7eb3f-751c-4ad1-a44a-c4e78bb73de9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:46.626758 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-84f7eb3f-751c-4ad1-a44a-c4e78bb73de9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:46.627961 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-84f7eb3f-751c-4ad1-a44a-c4e78bb73de9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1808 microversion: 2.20 time: 0.158103 Oct 30 01:01:46.628528 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 366/734] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:01:46 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1808 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:01:47.227304 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-0e9b60f9-76d7-4ae6-8f82-fb676579c91e tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:47.228197 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-0e9b60f9-76d7-4ae6-8f82-fb676579c91e tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: db9ac414-31a5-4570-aa3f-37db3dac2049] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:47.237004 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0e9b60f9-76d7-4ae6-8f82-fb676579c91e tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:47.237694 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0e9b60f9-76d7-4ae6-8f82-fb676579c91e tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:47.238311 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0e9b60f9-76d7-4ae6-8f82-fb676579c91e tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:47.239294 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0e9b60f9-76d7-4ae6-8f82-fb676579c91e tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:47.240111 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0e9b60f9-76d7-4ae6-8f82-fb676579c91e tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:47.240608 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0e9b60f9-76d7-4ae6-8f82-fb676579c91e tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:47.375313 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-0e9b60f9-76d7-4ae6-8f82-fb676579c91e tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:47.383738 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-0e9b60f9-76d7-4ae6-8f82-fb676579c91e tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049" status: 200 len: 1515 microversion: 2.1 time: 0.162037 Oct 30 01:01:47.384326 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 369/735] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:47 2022] GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049 => generated 1515 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:47.453498 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-eea641ff-a726-4440-8171-cee97bbab5c1 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:47.454715 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-eea641ff-a726-4440-8171-cee97bbab5c1 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5632075c-7062-434e-b00c-bbf04c5ac04a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:47.462846 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-eea641ff-a726-4440-8171-cee97bbab5c1 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:47.463321 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-eea641ff-a726-4440-8171-cee97bbab5c1 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:47.463753 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-eea641ff-a726-4440-8171-cee97bbab5c1 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:47.464464 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-eea641ff-a726-4440-8171-cee97bbab5c1 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:47.464902 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-eea641ff-a726-4440-8171-cee97bbab5c1 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:47.465309 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-eea641ff-a726-4440-8171-cee97bbab5c1 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:47.614738 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-eea641ff-a726-4440-8171-cee97bbab5c1 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:47.622562 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-eea641ff-a726-4440-8171-cee97bbab5c1 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a" status: 200 len: 1366 microversion: 2.1 time: 0.174579 Oct 30 01:01:47.623101 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 367/736] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:47 2022] GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a => generated 1366 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:47.656421 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-9df01a26-8cb2-4fb4-8cbe-0052f78ea29e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:47.657205 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-9df01a26-8cb2-4fb4-8cbe-0052f78ea29e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:47.664851 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9df01a26-8cb2-4fb4-8cbe-0052f78ea29e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:47.665338 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9df01a26-8cb2-4fb4-8cbe-0052f78ea29e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:47.665772 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9df01a26-8cb2-4fb4-8cbe-0052f78ea29e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:47.666463 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9df01a26-8cb2-4fb4-8cbe-0052f78ea29e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:47.666915 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9df01a26-8cb2-4fb4-8cbe-0052f78ea29e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:47.667337 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9df01a26-8cb2-4fb4-8cbe-0052f78ea29e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:47.800880 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-9df01a26-8cb2-4fb4-8cbe-0052f78ea29e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:47.810186 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-9df01a26-8cb2-4fb4-8cbe-0052f78ea29e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:47.811540 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-9df01a26-8cb2-4fb4-8cbe-0052f78ea29e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:47.812788 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-9df01a26-8cb2-4fb4-8cbe-0052f78ea29e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1808 microversion: 2.20 time: 0.162187 Oct 30 01:01:47.813349 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 370/737] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:01:47 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1808 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:01:48.407630 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-d609c84e-d571-4601-b082-3ea5f9528d5b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:48.408256 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-d609c84e-d571-4601-b082-3ea5f9528d5b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: db9ac414-31a5-4570-aa3f-37db3dac2049] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:48.414682 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d609c84e-d571-4601-b082-3ea5f9528d5b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:48.415160 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d609c84e-d571-4601-b082-3ea5f9528d5b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:48.415586 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d609c84e-d571-4601-b082-3ea5f9528d5b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:48.416296 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d609c84e-d571-4601-b082-3ea5f9528d5b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:48.416739 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d609c84e-d571-4601-b082-3ea5f9528d5b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:48.417149 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d609c84e-d571-4601-b082-3ea5f9528d5b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:48.564209 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-d609c84e-d571-4601-b082-3ea5f9528d5b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:48.572891 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-d609c84e-d571-4601-b082-3ea5f9528d5b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049" status: 200 len: 1515 microversion: 2.1 time: 0.169133 Oct 30 01:01:48.573439 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 368/738] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:48 2022] GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049 => generated 1515 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:48.648783 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-cd5c24c5-9f63-4a3b-a524-dafd069a065e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:48.649402 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-cd5c24c5-9f63-4a3b-a524-dafd069a065e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5632075c-7062-434e-b00c-bbf04c5ac04a] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:48.657416 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cd5c24c5-9f63-4a3b-a524-dafd069a065e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:48.658063 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cd5c24c5-9f63-4a3b-a524-dafd069a065e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:48.658605 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cd5c24c5-9f63-4a3b-a524-dafd069a065e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:48.659426 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cd5c24c5-9f63-4a3b-a524-dafd069a065e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:48.660013 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cd5c24c5-9f63-4a3b-a524-dafd069a065e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:48.660511 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cd5c24c5-9f63-4a3b-a524-dafd069a065e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:48.797731 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-cd5c24c5-9f63-4a3b-a524-dafd069a065e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:48.805900 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-cd5c24c5-9f63-4a3b-a524-dafd069a065e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a" status: 200 len: 1366 microversion: 2.1 time: 0.162839 Oct 30 01:01:48.806474 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 371/739] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:48 2022] GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a => generated 1366 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:48.838668 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-abbff1a7-f028-451a-9cf0-881c43897ea5 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:48.839265 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-abbff1a7-f028-451a-9cf0-881c43897ea5 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:48.845428 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-abbff1a7-f028-451a-9cf0-881c43897ea5 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:48.845923 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-abbff1a7-f028-451a-9cf0-881c43897ea5 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:48.846341 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-abbff1a7-f028-451a-9cf0-881c43897ea5 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:48.847008 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-abbff1a7-f028-451a-9cf0-881c43897ea5 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:48.847562 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-abbff1a7-f028-451a-9cf0-881c43897ea5 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:48.848016 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-abbff1a7-f028-451a-9cf0-881c43897ea5 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:48.980042 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-abbff1a7-f028-451a-9cf0-881c43897ea5 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:48.989020 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-abbff1a7-f028-451a-9cf0-881c43897ea5 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:48.990498 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-abbff1a7-f028-451a-9cf0-881c43897ea5 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:48.991599 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-abbff1a7-f028-451a-9cf0-881c43897ea5 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1808 microversion: 2.20 time: 0.156855 Oct 30 01:01:48.992166 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 369/740] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:01:48 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1808 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:01:49.601054 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-fd766a74-25b0-4e00-b438-64a15dd400e1 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:49.602067 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-fd766a74-25b0-4e00-b438-64a15dd400e1 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: db9ac414-31a5-4570-aa3f-37db3dac2049] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:49.612370 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-fd766a74-25b0-4e00-b438-64a15dd400e1 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:49.613068 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-fd766a74-25b0-4e00-b438-64a15dd400e1 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:49.613849 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-fd766a74-25b0-4e00-b438-64a15dd400e1 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:49.615001 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-fd766a74-25b0-4e00-b438-64a15dd400e1 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:49.615681 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-fd766a74-25b0-4e00-b438-64a15dd400e1 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:49.616143 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-fd766a74-25b0-4e00-b438-64a15dd400e1 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:49.754551 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-fd766a74-25b0-4e00-b438-64a15dd400e1 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:49.762921 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-fd766a74-25b0-4e00-b438-64a15dd400e1 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049" status: 200 len: 1515 microversion: 2.1 time: 0.168419 Oct 30 01:01:49.763535 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 372/741] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:49 2022] GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049 => generated 1515 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:49.834357 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-1777139a-ec51-41cf-bbe2-6bee988482e3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:49.835368 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-1777139a-ec51-41cf-bbe2-6bee988482e3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5632075c-7062-434e-b00c-bbf04c5ac04a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:49.846463 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1777139a-ec51-41cf-bbe2-6bee988482e3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:49.847374 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1777139a-ec51-41cf-bbe2-6bee988482e3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:49.848058 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1777139a-ec51-41cf-bbe2-6bee988482e3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:49.849499 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1777139a-ec51-41cf-bbe2-6bee988482e3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:49.849935 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1777139a-ec51-41cf-bbe2-6bee988482e3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:49.850371 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1777139a-ec51-41cf-bbe2-6bee988482e3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:49.989060 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-1777139a-ec51-41cf-bbe2-6bee988482e3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:49.997159 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-1777139a-ec51-41cf-bbe2-6bee988482e3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a" status: 200 len: 1366 microversion: 2.1 time: 0.170235 Oct 30 01:01:49.997730 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 370/742] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:49 2022] GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a => generated 1366 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:50.017034 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-32b795b2-8cf9-4fcf-8df8-0a0e41ba2977 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:50.017653 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-32b795b2-8cf9-4fcf-8df8-0a0e41ba2977 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:50.023777 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-32b795b2-8cf9-4fcf-8df8-0a0e41ba2977 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:50.024244 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-32b795b2-8cf9-4fcf-8df8-0a0e41ba2977 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:50.024677 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-32b795b2-8cf9-4fcf-8df8-0a0e41ba2977 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:50.025361 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-32b795b2-8cf9-4fcf-8df8-0a0e41ba2977 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:50.025812 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-32b795b2-8cf9-4fcf-8df8-0a0e41ba2977 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:50.026226 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-32b795b2-8cf9-4fcf-8df8-0a0e41ba2977 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:50.167039 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-32b795b2-8cf9-4fcf-8df8-0a0e41ba2977 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:50.176001 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-32b795b2-8cf9-4fcf-8df8-0a0e41ba2977 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:50.177513 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-32b795b2-8cf9-4fcf-8df8-0a0e41ba2977 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:50.178867 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-32b795b2-8cf9-4fcf-8df8-0a0e41ba2977 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1808 microversion: 2.20 time: 0.165795 Oct 30 01:01:50.179421 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 373/743] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:01:50 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1808 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:01:50.786969 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-d34ccf4a-1f4b-4b73-bd2b-6a88f9c978d1 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:50.787942 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-d34ccf4a-1f4b-4b73-bd2b-6a88f9c978d1 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: db9ac414-31a5-4570-aa3f-37db3dac2049] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:50.794323 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d34ccf4a-1f4b-4b73-bd2b-6a88f9c978d1 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:50.794848 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d34ccf4a-1f4b-4b73-bd2b-6a88f9c978d1 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:50.795274 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d34ccf4a-1f4b-4b73-bd2b-6a88f9c978d1 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:50.795987 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d34ccf4a-1f4b-4b73-bd2b-6a88f9c978d1 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:50.796814 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d34ccf4a-1f4b-4b73-bd2b-6a88f9c978d1 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:50.797233 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d34ccf4a-1f4b-4b73-bd2b-6a88f9c978d1 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:50.935867 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-d34ccf4a-1f4b-4b73-bd2b-6a88f9c978d1 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:50.944118 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-d34ccf4a-1f4b-4b73-bd2b-6a88f9c978d1 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049" status: 200 len: 1515 microversion: 2.1 time: 0.161173 Oct 30 01:01:50.944841 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 371/744] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:50 2022] GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049 => generated 1515 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:51.020825 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-199bebcf-c5a5-46bc-a038-94d7c9bac816 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:51.021469 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-199bebcf-c5a5-46bc-a038-94d7c9bac816 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5632075c-7062-434e-b00c-bbf04c5ac04a] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:51.023941 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_api:SELECT=77 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:01:51.029341 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-199bebcf-c5a5-46bc-a038-94d7c9bac816 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:51.029826 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-199bebcf-c5a5-46bc-a038-94d7c9bac816 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:51.030262 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-199bebcf-c5a5-46bc-a038-94d7c9bac816 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:51.030966 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-199bebcf-c5a5-46bc-a038-94d7c9bac816 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:51.031420 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-199bebcf-c5a5-46bc-a038-94d7c9bac816 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:51.031872 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-199bebcf-c5a5-46bc-a038-94d7c9bac816 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:51.171083 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-199bebcf-c5a5-46bc-a038-94d7c9bac816 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:51.179904 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-199bebcf-c5a5-46bc-a038-94d7c9bac816 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a" status: 200 len: 1366 microversion: 2.1 time: 0.163205 Oct 30 01:01:51.180513 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 374/745] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:51 2022] GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a => generated 1366 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:51.204622 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-10ea2897-2714-4189-8575-4867a80897e0 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:51.205234 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-10ea2897-2714-4189-8575-4867a80897e0 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:51.211392 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-10ea2897-2714-4189-8575-4867a80897e0 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:51.211912 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-10ea2897-2714-4189-8575-4867a80897e0 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:51.212317 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-10ea2897-2714-4189-8575-4867a80897e0 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:51.213002 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-10ea2897-2714-4189-8575-4867a80897e0 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:51.213445 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-10ea2897-2714-4189-8575-4867a80897e0 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:51.213859 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-10ea2897-2714-4189-8575-4867a80897e0 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:51.359745 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-10ea2897-2714-4189-8575-4867a80897e0 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:51.369324 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-10ea2897-2714-4189-8575-4867a80897e0 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:51.370658 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-10ea2897-2714-4189-8575-4867a80897e0 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:51.371843 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-10ea2897-2714-4189-8575-4867a80897e0 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1808 microversion: 2.20 time: 0.171519 Oct 30 01:01:51.372400 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 372/746] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:01:51 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1808 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:01:51.971582 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-86dbbb1e-cbfe-42fb-a392-5ca183eabdbb tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:51.974884 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-86dbbb1e-cbfe-42fb-a392-5ca183eabdbb tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: db9ac414-31a5-4570-aa3f-37db3dac2049] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:51.979289 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-86dbbb1e-cbfe-42fb-a392-5ca183eabdbb tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:51.979772 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-86dbbb1e-cbfe-42fb-a392-5ca183eabdbb tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:51.980233 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-86dbbb1e-cbfe-42fb-a392-5ca183eabdbb tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:51.980918 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-86dbbb1e-cbfe-42fb-a392-5ca183eabdbb tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:51.981363 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-86dbbb1e-cbfe-42fb-a392-5ca183eabdbb tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:51.981778 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-86dbbb1e-cbfe-42fb-a392-5ca183eabdbb tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:52.120521 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-86dbbb1e-cbfe-42fb-a392-5ca183eabdbb tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:52.128871 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-86dbbb1e-cbfe-42fb-a392-5ca183eabdbb tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049" status: 200 len: 1515 microversion: 2.1 time: 0.163703 Oct 30 01:01:52.131023 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 375/747] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:51 2022] GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049 => generated 1515 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:52.203000 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-b53489f3-9f39-47c7-9d35-3adeae9a9580 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:52.203799 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-b53489f3-9f39-47c7-9d35-3adeae9a9580 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5632075c-7062-434e-b00c-bbf04c5ac04a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:52.211887 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b53489f3-9f39-47c7-9d35-3adeae9a9580 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:52.212305 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b53489f3-9f39-47c7-9d35-3adeae9a9580 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:52.212771 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b53489f3-9f39-47c7-9d35-3adeae9a9580 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:52.213479 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b53489f3-9f39-47c7-9d35-3adeae9a9580 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:52.213951 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b53489f3-9f39-47c7-9d35-3adeae9a9580 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:52.214368 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b53489f3-9f39-47c7-9d35-3adeae9a9580 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:52.347605 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-b53489f3-9f39-47c7-9d35-3adeae9a9580 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:52.356008 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-b53489f3-9f39-47c7-9d35-3adeae9a9580 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a" status: 200 len: 1556 microversion: 2.1 time: 0.157345 Oct 30 01:01:52.356589 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 373/748] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:52 2022] GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a => generated 1556 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 01:01:52.378597 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-2aee9bf1-a299-40cd-8c2e-2bf48d9ce47f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:52.379086 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-2aee9bf1-a299-40cd-8c2e-2bf48d9ce47f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5632075c-7062-434e-b00c-bbf04c5ac04a] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:52.385014 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2aee9bf1-a299-40cd-8c2e-2bf48d9ce47f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:52.385476 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2aee9bf1-a299-40cd-8c2e-2bf48d9ce47f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:52.385903 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2aee9bf1-a299-40cd-8c2e-2bf48d9ce47f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:52.386584 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2aee9bf1-a299-40cd-8c2e-2bf48d9ce47f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:52.387024 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2aee9bf1-a299-40cd-8c2e-2bf48d9ce47f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:52.387436 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2aee9bf1-a299-40cd-8c2e-2bf48d9ce47f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:52.397852 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-0e0e194e-3c4b-43bf-a3a1-278497cb1a01 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:52.398530 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-0e0e194e-3c4b-43bf-a3a1-278497cb1a01 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:52.404899 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0e0e194e-3c4b-43bf-a3a1-278497cb1a01 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:52.405083 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0e0e194e-3c4b-43bf-a3a1-278497cb1a01 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:52.405436 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0e0e194e-3c4b-43bf-a3a1-278497cb1a01 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:52.406129 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0e0e194e-3c4b-43bf-a3a1-278497cb1a01 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:52.406721 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0e0e194e-3c4b-43bf-a3a1-278497cb1a01 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:52.407150 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0e0e194e-3c4b-43bf-a3a1-278497cb1a01 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:52.502715 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-2aee9bf1-a299-40cd-8c2e-2bf48d9ce47f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.127853 Oct 30 01:01:52.504113 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 376/749] 10.222.0.16 () {58 vars in 1275 bytes} [Sun Oct 30 01:01:52 2022] GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a/os-interface => generated 284 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:01:52.662508 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-0e0e194e-3c4b-43bf-a3a1-278497cb1a01 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:52.672898 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-0e0e194e-3c4b-43bf-a3a1-278497cb1a01 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:52.674385 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-0e0e194e-3c4b-43bf-a3a1-278497cb1a01 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:52.675679 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-0e0e194e-3c4b-43bf-a3a1-278497cb1a01 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1808 microversion: 2.20 time: 0.282318 Oct 30 01:01:52.676767 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 374/750] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:01:52 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1808 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:01:53.154052 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-460f3da1-b42c-44df-8a6e-9d20c5bd1d97 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:53.156811 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-460f3da1-b42c-44df-8a6e-9d20c5bd1d97 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: db9ac414-31a5-4570-aa3f-37db3dac2049] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:53.165931 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-460f3da1-b42c-44df-8a6e-9d20c5bd1d97 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:53.166451 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-460f3da1-b42c-44df-8a6e-9d20c5bd1d97 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:53.166868 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-460f3da1-b42c-44df-8a6e-9d20c5bd1d97 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:53.167804 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-460f3da1-b42c-44df-8a6e-9d20c5bd1d97 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:53.168234 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-460f3da1-b42c-44df-8a6e-9d20c5bd1d97 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:53.169245 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-460f3da1-b42c-44df-8a6e-9d20c5bd1d97 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:53.419480 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [req-4d468677-087f-41d2-8813-ae20cdf48261 req-1753d65a-4644-44bf-b3db-e99842d1c2b3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "db9ac414-31a5-4570-aa3f-37db3dac2049", "name": "network-vif-deleted", "tag": "56471ebd-e8c3-4ce0-bd02-87518e117371"}]} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:01:53.433758 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-4d468677-087f-41d2-8813-ae20cdf48261 req-1753d65a-4644-44bf-b3db-e99842d1c2b3 service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:53.434241 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-4d468677-087f-41d2-8813-ae20cdf48261 req-1753d65a-4644-44bf-b3db-e99842d1c2b3 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:53.434698 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-4d468677-087f-41d2-8813-ae20cdf48261 req-1753d65a-4644-44bf-b3db-e99842d1c2b3 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:53.449978 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.compute.server_external_events [req-4d468677-087f-41d2-8813-ae20cdf48261 req-1753d65a-4644-44bf-b3db-e99842d1c2b3 service nova] Creating event network-vif-deleted:56471ebd-e8c3-4ce0-bd02-87518e117371 for instance db9ac414-31a5-4570-aa3f-37db3dac2049 on hv-cinder-maste Oct 30 01:01:53.456041 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [req-4d468677-087f-41d2-8813-ae20cdf48261 req-1753d65a-4644-44bf-b3db-e99842d1c2b3 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043689 Oct 30 01:01:53.456909 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 375/751] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:01:53 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:01:53.521304 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-460f3da1-b42c-44df-8a6e-9d20c5bd1d97 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:53.529298 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-460f3da1-b42c-44df-8a6e-9d20c5bd1d97 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049" status: 200 len: 1473 microversion: 2.1 time: 0.379975 Oct 30 01:01:53.529886 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 377/752] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:53 2022] GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049 => generated 1473 bytes in 381 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:53.701476 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-1c67afaa-96d5-4449-add2-a877238a1a46 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:53.702170 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-1c67afaa-96d5-4449-add2-a877238a1a46 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:53.708752 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1c67afaa-96d5-4449-add2-a877238a1a46 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:53.709492 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1c67afaa-96d5-4449-add2-a877238a1a46 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:53.709920 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1c67afaa-96d5-4449-add2-a877238a1a46 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:53.710621 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1c67afaa-96d5-4449-add2-a877238a1a46 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:53.711070 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1c67afaa-96d5-4449-add2-a877238a1a46 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:53.711481 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1c67afaa-96d5-4449-add2-a877238a1a46 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:53.935440 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [req-d7f534a5-be3e-494b-860c-fa3e7e222d81 req-54abeb18-d497-486d-8a69-c4655dce768e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5632075c-7062-434e-b00c-bbf04c5ac04a", "tag": "c9fe8633-400b-4adf-99ff-00c04190ea67"}]} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:01:53.953984 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-1c67afaa-96d5-4449-add2-a877238a1a46 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:53.967652 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-1c67afaa-96d5-4449-add2-a877238a1a46 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:53.969206 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-d7f534a5-be3e-494b-860c-fa3e7e222d81 req-54abeb18-d497-486d-8a69-c4655dce768e service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:53.969675 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-1c67afaa-96d5-4449-add2-a877238a1a46 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:53.969818 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-d7f534a5-be3e-494b-860c-fa3e7e222d81 req-54abeb18-d497-486d-8a69-c4655dce768e service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:53.970107 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-d7f534a5-be3e-494b-860c-fa3e7e222d81 req-54abeb18-d497-486d-8a69-c4655dce768e service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:53.971459 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-1c67afaa-96d5-4449-add2-a877238a1a46 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1808 microversion: 2.20 time: 0.274403 Oct 30 01:01:53.973044 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 376/753] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:01:53 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1808 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:01:53.995973 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.compute.server_external_events [req-d7f534a5-be3e-494b-860c-fa3e7e222d81 req-54abeb18-d497-486d-8a69-c4655dce768e service nova] Creating event network-changed:c9fe8633-400b-4adf-99ff-00c04190ea67 for instance 5632075c-7062-434e-b00c-bbf04c5ac04a on hv-cinder-maste Oct 30 01:01:54.002016 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [req-d7f534a5-be3e-494b-860c-fa3e7e222d81 req-54abeb18-d497-486d-8a69-c4655dce768e service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.150600 Oct 30 01:01:54.003106 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 378/754] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:01:53 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 30 01:01:54.558834 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-636913a1-34ef-4f15-bb9b-22aa76504408 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:54.559633 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-636913a1-34ef-4f15-bb9b-22aa76504408 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: db9ac414-31a5-4570-aa3f-37db3dac2049] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:54.571863 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-636913a1-34ef-4f15-bb9b-22aa76504408 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:54.572656 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-636913a1-34ef-4f15-bb9b-22aa76504408 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:54.573213 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-636913a1-34ef-4f15-bb9b-22aa76504408 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:54.574028 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-636913a1-34ef-4f15-bb9b-22aa76504408 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:54.574581 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-636913a1-34ef-4f15-bb9b-22aa76504408 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:54.575095 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-636913a1-34ef-4f15-bb9b-22aa76504408 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:54.636111 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-636913a1-34ef-4f15-bb9b-22aa76504408 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:54.636669 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-636913a1-34ef-4f15-bb9b-22aa76504408 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:54.637215 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-636913a1-34ef-4f15-bb9b-22aa76504408 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:54.637782 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.objects.instance [None req-636913a1-34ef-4f15-bb9b-22aa76504408 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lazy-loading 'fault' on Instance uuid db9ac414-31a5-4570-aa3f-37db3dac2049 {{(pid=118962) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 30 01:01:54.686372 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-636913a1-34ef-4f15-bb9b-22aa76504408 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:54.694787 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-636913a1-34ef-4f15-bb9b-22aa76504408 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049" status: 200 len: 1309 microversion: 2.1 time: 0.140572 Oct 30 01:01:54.695507 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 377/755] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:54 2022] GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049 => generated 1309 bytes in 142 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:54.999667 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-38ee4fd6-fcd2-462a-b149-b7f2c121527c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:55.000386 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-38ee4fd6-fcd2-462a-b149-b7f2c121527c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:55.007382 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-38ee4fd6-fcd2-462a-b149-b7f2c121527c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:55.007928 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-38ee4fd6-fcd2-462a-b149-b7f2c121527c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:55.008333 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-38ee4fd6-fcd2-462a-b149-b7f2c121527c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:55.009054 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-38ee4fd6-fcd2-462a-b149-b7f2c121527c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:55.009690 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-38ee4fd6-fcd2-462a-b149-b7f2c121527c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:55.010305 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-38ee4fd6-fcd2-462a-b149-b7f2c121527c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:55.211059 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-38ee4fd6-fcd2-462a-b149-b7f2c121527c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:55.220098 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-38ee4fd6-fcd2-462a-b149-b7f2c121527c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:55.221845 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-38ee4fd6-fcd2-462a-b149-b7f2c121527c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:55.223236 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-38ee4fd6-fcd2-462a-b149-b7f2c121527c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1808 microversion: 2.20 time: 0.229175 Oct 30 01:01:55.223820 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 379/756] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:01:54 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1808 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:01:55.719558 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-19bc208e-8a31-415f-8ae2-3d3332884bd9 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:55.720302 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-19bc208e-8a31-415f-8ae2-3d3332884bd9 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: db9ac414-31a5-4570-aa3f-37db3dac2049] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:55.727979 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-19bc208e-8a31-415f-8ae2-3d3332884bd9 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:55.728580 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-19bc208e-8a31-415f-8ae2-3d3332884bd9 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:55.729106 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-19bc208e-8a31-415f-8ae2-3d3332884bd9 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:55.730240 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-19bc208e-8a31-415f-8ae2-3d3332884bd9 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:55.730931 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-19bc208e-8a31-415f-8ae2-3d3332884bd9 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:55.731759 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-19bc208e-8a31-415f-8ae2-3d3332884bd9 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:55.770136 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-19bc208e-8a31-415f-8ae2-3d3332884bd9 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:55.770837 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-19bc208e-8a31-415f-8ae2-3d3332884bd9 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:55.771440 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-19bc208e-8a31-415f-8ae2-3d3332884bd9 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:55.772088 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.objects.instance [None req-19bc208e-8a31-415f-8ae2-3d3332884bd9 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lazy-loading 'fault' on Instance uuid db9ac414-31a5-4570-aa3f-37db3dac2049 {{(pid=118962) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 30 01:01:55.814310 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-19bc208e-8a31-415f-8ae2-3d3332884bd9 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:55.822222 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-19bc208e-8a31-415f-8ae2-3d3332884bd9 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049" status: 200 len: 1309 microversion: 2.1 time: 0.107141 Oct 30 01:01:55.822795 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 378/757] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:55 2022] GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049 => generated 1309 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:56.253362 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-92897c48-36b0-40b8-97b6-a48f40d58ff2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:56.254035 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-92897c48-36b0-40b8-97b6-a48f40d58ff2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:56.263788 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-92897c48-36b0-40b8-97b6-a48f40d58ff2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:56.264511 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-92897c48-36b0-40b8-97b6-a48f40d58ff2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:56.265105 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-92897c48-36b0-40b8-97b6-a48f40d58ff2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:56.266135 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-92897c48-36b0-40b8-97b6-a48f40d58ff2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:56.266773 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-92897c48-36b0-40b8-97b6-a48f40d58ff2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:56.267402 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-92897c48-36b0-40b8-97b6-a48f40d58ff2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:56.409954 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-92897c48-36b0-40b8-97b6-a48f40d58ff2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:56.423566 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-92897c48-36b0-40b8-97b6-a48f40d58ff2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:56.425781 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-92897c48-36b0-40b8-97b6-a48f40d58ff2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:56.427568 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-92897c48-36b0-40b8-97b6-a48f40d58ff2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1808 microversion: 2.20 time: 0.180327 Oct 30 01:01:56.428359 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 380/758] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:01:56 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1808 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:01:56.845531 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-11cf4861-7c50-4a2e-9eb1-61125b39dcdd tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:56.846335 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-11cf4861-7c50-4a2e-9eb1-61125b39dcdd tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: db9ac414-31a5-4570-aa3f-37db3dac2049] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:56.853610 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-11cf4861-7c50-4a2e-9eb1-61125b39dcdd tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:56.854404 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-11cf4861-7c50-4a2e-9eb1-61125b39dcdd tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:56.855055 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-11cf4861-7c50-4a2e-9eb1-61125b39dcdd tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:56.856008 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-11cf4861-7c50-4a2e-9eb1-61125b39dcdd tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:56.856657 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-11cf4861-7c50-4a2e-9eb1-61125b39dcdd tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:56.857462 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-11cf4861-7c50-4a2e-9eb1-61125b39dcdd tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:56.891678 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-11cf4861-7c50-4a2e-9eb1-61125b39dcdd tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:56.892254 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-11cf4861-7c50-4a2e-9eb1-61125b39dcdd tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:56.892758 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-11cf4861-7c50-4a2e-9eb1-61125b39dcdd tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:56.893240 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.objects.instance [None req-11cf4861-7c50-4a2e-9eb1-61125b39dcdd tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lazy-loading 'fault' on Instance uuid db9ac414-31a5-4570-aa3f-37db3dac2049 {{(pid=118962) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 30 01:01:56.959741 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-11cf4861-7c50-4a2e-9eb1-61125b39dcdd tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:56.968585 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-11cf4861-7c50-4a2e-9eb1-61125b39dcdd tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049" status: 200 len: 1309 microversion: 2.1 time: 0.126993 Oct 30 01:01:56.969186 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 379/759] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:56 2022] GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049 => generated 1309 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:57.455532 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-55157dfd-acbb-496b-aae1-422f2f94db6e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:57.456248 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-55157dfd-acbb-496b-aae1-422f2f94db6e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:57.472513 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-55157dfd-acbb-496b-aae1-422f2f94db6e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:57.473069 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-55157dfd-acbb-496b-aae1-422f2f94db6e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:57.473433 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-55157dfd-acbb-496b-aae1-422f2f94db6e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:57.474133 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-55157dfd-acbb-496b-aae1-422f2f94db6e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:57.474665 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-55157dfd-acbb-496b-aae1-422f2f94db6e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:57.475086 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-55157dfd-acbb-496b-aae1-422f2f94db6e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:57.479512 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell1:SELECT=100 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:01:57.632329 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-55157dfd-acbb-496b-aae1-422f2f94db6e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:57.641878 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-55157dfd-acbb-496b-aae1-422f2f94db6e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:57.643308 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-55157dfd-acbb-496b-aae1-422f2f94db6e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:57.644536 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-55157dfd-acbb-496b-aae1-422f2f94db6e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1808 microversion: 2.20 time: 0.196079 Oct 30 01:01:57.645078 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 381/760] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:01:57 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1808 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:01:57.998373 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-96b27eb1-0298-40f4-83fd-7478743287ad tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:58.001271 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-96b27eb1-0298-40f4-83fd-7478743287ad tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: db9ac414-31a5-4570-aa3f-37db3dac2049] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:58.010789 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-96b27eb1-0298-40f4-83fd-7478743287ad tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:58.011274 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-96b27eb1-0298-40f4-83fd-7478743287ad tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:58.011703 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-96b27eb1-0298-40f4-83fd-7478743287ad tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:58.012418 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-96b27eb1-0298-40f4-83fd-7478743287ad tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:58.012861 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-96b27eb1-0298-40f4-83fd-7478743287ad tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:58.013272 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-96b27eb1-0298-40f4-83fd-7478743287ad tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:58.050702 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-96b27eb1-0298-40f4-83fd-7478743287ad tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:58.051182 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-96b27eb1-0298-40f4-83fd-7478743287ad tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:58.051607 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-96b27eb1-0298-40f4-83fd-7478743287ad tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:58.052187 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.objects.instance [None req-96b27eb1-0298-40f4-83fd-7478743287ad tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lazy-loading 'fault' on Instance uuid db9ac414-31a5-4570-aa3f-37db3dac2049 {{(pid=118962) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 30 01:01:58.095192 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-96b27eb1-0298-40f4-83fd-7478743287ad tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:58.103214 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-96b27eb1-0298-40f4-83fd-7478743287ad tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049" status: 200 len: 1309 microversion: 2.1 time: 0.113994 Oct 30 01:01:58.104059 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 380/761] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:57 2022] GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049 => generated 1309 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:58.676894 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-840e5a8a-898b-4a6d-84c7-d1d7800fa2a4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:58.677544 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-840e5a8a-898b-4a6d-84c7-d1d7800fa2a4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:58.684386 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-840e5a8a-898b-4a6d-84c7-d1d7800fa2a4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:58.684874 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-840e5a8a-898b-4a6d-84c7-d1d7800fa2a4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:58.685309 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-840e5a8a-898b-4a6d-84c7-d1d7800fa2a4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:58.686007 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-840e5a8a-898b-4a6d-84c7-d1d7800fa2a4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:58.686467 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-840e5a8a-898b-4a6d-84c7-d1d7800fa2a4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:58.686888 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-840e5a8a-898b-4a6d-84c7-d1d7800fa2a4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:58.834539 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-840e5a8a-898b-4a6d-84c7-d1d7800fa2a4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:58.843049 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-840e5a8a-898b-4a6d-84c7-d1d7800fa2a4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:58.844498 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-840e5a8a-898b-4a6d-84c7-d1d7800fa2a4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:58.845654 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-840e5a8a-898b-4a6d-84c7-d1d7800fa2a4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1808 microversion: 2.20 time: 0.175519 Oct 30 01:01:58.846226 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 382/762] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:01:58 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1808 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:01:59.126635 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-997df957-15b2-4e1b-a1f6-02b2330d84c3 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:59.127249 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-997df957-15b2-4e1b-a1f6-02b2330d84c3 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: db9ac414-31a5-4570-aa3f-37db3dac2049] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:59.134212 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-997df957-15b2-4e1b-a1f6-02b2330d84c3 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:59.134688 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-997df957-15b2-4e1b-a1f6-02b2330d84c3 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:59.135118 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-997df957-15b2-4e1b-a1f6-02b2330d84c3 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:59.135827 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-997df957-15b2-4e1b-a1f6-02b2330d84c3 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:59.136272 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-997df957-15b2-4e1b-a1f6-02b2330d84c3 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:59.136680 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-997df957-15b2-4e1b-a1f6-02b2330d84c3 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:59.169181 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-997df957-15b2-4e1b-a1f6-02b2330d84c3 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:59.169669 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-997df957-15b2-4e1b-a1f6-02b2330d84c3 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:59.170102 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-997df957-15b2-4e1b-a1f6-02b2330d84c3 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:59.170534 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.objects.instance [None req-997df957-15b2-4e1b-a1f6-02b2330d84c3 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lazy-loading 'fault' on Instance uuid db9ac414-31a5-4570-aa3f-37db3dac2049 {{(pid=118962) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 30 01:01:59.217470 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-997df957-15b2-4e1b-a1f6-02b2330d84c3 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:01:59.222486 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell1:SELECT=100 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:01:59.230103 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-997df957-15b2-4e1b-a1f6-02b2330d84c3 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049" status: 200 len: 1309 microversion: 2.1 time: 0.107073 Oct 30 01:01:59.230826 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 381/763] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:01:59 2022] GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049 => generated 1309 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:01:59.870900 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-3a20dc3f-4ff7-4b07-8c61-58daa2cec80b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:01:59.871560 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-3a20dc3f-4ff7-4b07-8c61-58daa2cec80b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:01:59.878204 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3a20dc3f-4ff7-4b07-8c61-58daa2cec80b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:59.878688 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3a20dc3f-4ff7-4b07-8c61-58daa2cec80b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:59.879120 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3a20dc3f-4ff7-4b07-8c61-58daa2cec80b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:01:59.879816 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3a20dc3f-4ff7-4b07-8c61-58daa2cec80b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:01:59.880296 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3a20dc3f-4ff7-4b07-8c61-58daa2cec80b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:01:59.880690 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3a20dc3f-4ff7-4b07-8c61-58daa2cec80b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:00.025986 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-3a20dc3f-4ff7-4b07-8c61-58daa2cec80b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:00.035604 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-3a20dc3f-4ff7-4b07-8c61-58daa2cec80b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:00.037279 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-3a20dc3f-4ff7-4b07-8c61-58daa2cec80b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:00.038581 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-3a20dc3f-4ff7-4b07-8c61-58daa2cec80b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1808 microversion: 2.20 time: 0.171733 Oct 30 01:02:00.039562 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 383/764] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:01:59 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1808 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:02:00.264291 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-d33d5880-d572-497f-ab4e-48562eaabe0a tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:02:00.264933 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-d33d5880-d572-497f-ab4e-48562eaabe0a tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: db9ac414-31a5-4570-aa3f-37db3dac2049] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:02:00.274923 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d33d5880-d572-497f-ab4e-48562eaabe0a tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:00.275520 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d33d5880-d572-497f-ab4e-48562eaabe0a tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:00.276023 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d33d5880-d572-497f-ab4e-48562eaabe0a tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:00.277033 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d33d5880-d572-497f-ab4e-48562eaabe0a tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:00.277713 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d33d5880-d572-497f-ab4e-48562eaabe0a tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:00.278310 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d33d5880-d572-497f-ab4e-48562eaabe0a tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:00.312816 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d33d5880-d572-497f-ab4e-48562eaabe0a tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:00.313298 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d33d5880-d572-497f-ab4e-48562eaabe0a tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:00.313727 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d33d5880-d572-497f-ab4e-48562eaabe0a tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:00.314169 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.objects.instance [None req-d33d5880-d572-497f-ab4e-48562eaabe0a tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lazy-loading 'fault' on Instance uuid db9ac414-31a5-4570-aa3f-37db3dac2049 {{(pid=118962) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 30 01:02:00.358155 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-d33d5880-d572-497f-ab4e-48562eaabe0a tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:00.371012 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-d33d5880-d572-497f-ab4e-48562eaabe0a tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049" status: 200 len: 1309 microversion: 2.1 time: 0.112803 Oct 30 01:02:00.371993 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 382/765] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:02:00 2022] GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049 => generated 1309 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:02:01.064600 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-68c3fa0c-9053-41d4-bde3-6cae21e204de tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:02:01.065249 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-68c3fa0c-9053-41d4-bde3-6cae21e204de tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:02:01.072230 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-68c3fa0c-9053-41d4-bde3-6cae21e204de tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:01.072717 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-68c3fa0c-9053-41d4-bde3-6cae21e204de tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:01.073140 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-68c3fa0c-9053-41d4-bde3-6cae21e204de tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:01.073866 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-68c3fa0c-9053-41d4-bde3-6cae21e204de tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:01.074296 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-68c3fa0c-9053-41d4-bde3-6cae21e204de tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:01.074715 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-68c3fa0c-9053-41d4-bde3-6cae21e204de tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:01.225641 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-68c3fa0c-9053-41d4-bde3-6cae21e204de tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:01.235186 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-68c3fa0c-9053-41d4-bde3-6cae21e204de tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:01.236664 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-68c3fa0c-9053-41d4-bde3-6cae21e204de tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:01.237856 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-68c3fa0c-9053-41d4-bde3-6cae21e204de tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1808 microversion: 2.20 time: 0.177356 Oct 30 01:02:01.238423 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 384/766] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:02:01 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1808 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:02:01.394741 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-f31d9d68-e687-4164-9f01-76618aba3a17 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:02:01.395375 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-f31d9d68-e687-4164-9f01-76618aba3a17 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: db9ac414-31a5-4570-aa3f-37db3dac2049] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:02:01.402704 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f31d9d68-e687-4164-9f01-76618aba3a17 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:01.403279 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f31d9d68-e687-4164-9f01-76618aba3a17 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:01.403834 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f31d9d68-e687-4164-9f01-76618aba3a17 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:01.404609 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f31d9d68-e687-4164-9f01-76618aba3a17 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:01.405143 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f31d9d68-e687-4164-9f01-76618aba3a17 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:01.405630 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f31d9d68-e687-4164-9f01-76618aba3a17 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:01.438070 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f31d9d68-e687-4164-9f01-76618aba3a17 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:01.438535 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f31d9d68-e687-4164-9f01-76618aba3a17 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:01.438952 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f31d9d68-e687-4164-9f01-76618aba3a17 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:01.439400 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.objects.instance [None req-f31d9d68-e687-4164-9f01-76618aba3a17 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lazy-loading 'fault' on Instance uuid db9ac414-31a5-4570-aa3f-37db3dac2049 {{(pid=118962) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 30 01:02:01.480703 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-f31d9d68-e687-4164-9f01-76618aba3a17 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:01.488489 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-f31d9d68-e687-4164-9f01-76618aba3a17 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049" status: 200 len: 1309 microversion: 2.1 time: 0.097729 Oct 30 01:02:01.489055 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 383/767] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:02:01 2022] GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049 => generated 1309 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:02:02.262850 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-10de530a-66a6-4e71-9a4f-ada67374e644 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:02:02.263437 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-10de530a-66a6-4e71-9a4f-ada67374e644 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:02:02.270457 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-10de530a-66a6-4e71-9a4f-ada67374e644 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:02.270934 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-10de530a-66a6-4e71-9a4f-ada67374e644 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:02.271378 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-10de530a-66a6-4e71-9a4f-ada67374e644 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:02.272263 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-10de530a-66a6-4e71-9a4f-ada67374e644 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:02.272689 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-10de530a-66a6-4e71-9a4f-ada67374e644 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:02.273110 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-10de530a-66a6-4e71-9a4f-ada67374e644 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:02.420820 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-10de530a-66a6-4e71-9a4f-ada67374e644 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:02.430831 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-10de530a-66a6-4e71-9a4f-ada67374e644 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:02.432445 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-10de530a-66a6-4e71-9a4f-ada67374e644 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:02.433784 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-10de530a-66a6-4e71-9a4f-ada67374e644 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1808 microversion: 2.20 time: 0.174913 Oct 30 01:02:02.434432 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 385/768] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:02:02 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1808 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:02:02.512171 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-b18639dd-d9e6-45c6-9f83-9433f4d296b6 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:02:02.512990 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-b18639dd-d9e6-45c6-9f83-9433f4d296b6 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: db9ac414-31a5-4570-aa3f-37db3dac2049] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:02:02.519606 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b18639dd-d9e6-45c6-9f83-9433f4d296b6 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:02.520183 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b18639dd-d9e6-45c6-9f83-9433f4d296b6 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:02.520649 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b18639dd-d9e6-45c6-9f83-9433f4d296b6 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:02.521563 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b18639dd-d9e6-45c6-9f83-9433f4d296b6 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:02.522025 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b18639dd-d9e6-45c6-9f83-9433f4d296b6 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:02.522441 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b18639dd-d9e6-45c6-9f83-9433f4d296b6 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:02.559301 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b18639dd-d9e6-45c6-9f83-9433f4d296b6 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:02.559810 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b18639dd-d9e6-45c6-9f83-9433f4d296b6 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:02.560245 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b18639dd-d9e6-45c6-9f83-9433f4d296b6 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:02.560707 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.objects.instance [None req-b18639dd-d9e6-45c6-9f83-9433f4d296b6 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lazy-loading 'fault' on Instance uuid db9ac414-31a5-4570-aa3f-37db3dac2049 {{(pid=118962) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 30 01:02:02.604361 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-b18639dd-d9e6-45c6-9f83-9433f4d296b6 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:02.612584 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-b18639dd-d9e6-45c6-9f83-9433f4d296b6 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049" status: 200 len: 1309 microversion: 2.1 time: 0.104500 Oct 30 01:02:02.613153 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 384/769] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:02:02 2022] GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049 => generated 1309 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:02:03.459226 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-85813da3-b397-469f-b21e-692763cacefd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:02:03.460268 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-85813da3-b397-469f-b21e-692763cacefd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:02:03.466840 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-85813da3-b397-469f-b21e-692763cacefd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:03.467323 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-85813da3-b397-469f-b21e-692763cacefd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:03.467745 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-85813da3-b397-469f-b21e-692763cacefd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:03.468497 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-85813da3-b397-469f-b21e-692763cacefd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:03.468972 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-85813da3-b397-469f-b21e-692763cacefd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:03.469398 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-85813da3-b397-469f-b21e-692763cacefd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:03.623784 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-85813da3-b397-469f-b21e-692763cacefd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:03.635381 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-2f9dc422-b307-4e71-90d9-82939b235cc2 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:02:03.636026 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-2f9dc422-b307-4e71-90d9-82939b235cc2 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: db9ac414-31a5-4570-aa3f-37db3dac2049] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:02:03.642337 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2f9dc422-b307-4e71-90d9-82939b235cc2 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:03.642826 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2f9dc422-b307-4e71-90d9-82939b235cc2 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:03.643250 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2f9dc422-b307-4e71-90d9-82939b235cc2 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:03.643974 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2f9dc422-b307-4e71-90d9-82939b235cc2 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:03.644418 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2f9dc422-b307-4e71-90d9-82939b235cc2 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:03.644839 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2f9dc422-b307-4e71-90d9-82939b235cc2 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:03.650699 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-85813da3-b397-469f-b21e-692763cacefd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:03.652161 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-85813da3-b397-469f-b21e-692763cacefd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:03.653356 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-85813da3-b397-469f-b21e-692763cacefd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1808 microversion: 2.20 time: 0.198100 Oct 30 01:02:03.653921 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 386/770] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:02:03 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1808 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:02:03.677302 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2f9dc422-b307-4e71-90d9-82939b235cc2 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:03.677778 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2f9dc422-b307-4e71-90d9-82939b235cc2 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:03.678205 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2f9dc422-b307-4e71-90d9-82939b235cc2 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:03.678919 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.objects.instance [None req-2f9dc422-b307-4e71-90d9-82939b235cc2 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lazy-loading 'fault' on Instance uuid db9ac414-31a5-4570-aa3f-37db3dac2049 {{(pid=118962) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 30 01:02:03.723755 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-2f9dc422-b307-4e71-90d9-82939b235cc2 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:03.733223 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-2f9dc422-b307-4e71-90d9-82939b235cc2 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049" status: 200 len: 1309 microversion: 2.1 time: 0.101346 Oct 30 01:02:03.734447 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 385/771] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:02:03 2022] GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049 => generated 1309 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:02:04.679252 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-4cfdee28-02d1-4371-8eaf-c450fb9e4201 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:02:04.679990 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-4cfdee28-02d1-4371-8eaf-c450fb9e4201 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:02:04.686717 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4cfdee28-02d1-4371-8eaf-c450fb9e4201 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:04.687208 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4cfdee28-02d1-4371-8eaf-c450fb9e4201 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:04.687658 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4cfdee28-02d1-4371-8eaf-c450fb9e4201 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:04.688390 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4cfdee28-02d1-4371-8eaf-c450fb9e4201 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:04.688845 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4cfdee28-02d1-4371-8eaf-c450fb9e4201 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:04.689241 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4cfdee28-02d1-4371-8eaf-c450fb9e4201 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:04.758110 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-b6f35804-725f-43c7-857d-8459bbeb6b3b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:02:04.758772 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-b6f35804-725f-43c7-857d-8459bbeb6b3b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: db9ac414-31a5-4570-aa3f-37db3dac2049] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:02:04.765261 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b6f35804-725f-43c7-857d-8459bbeb6b3b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:04.765747 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b6f35804-725f-43c7-857d-8459bbeb6b3b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:04.766178 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b6f35804-725f-43c7-857d-8459bbeb6b3b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:04.766882 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b6f35804-725f-43c7-857d-8459bbeb6b3b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:04.767332 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b6f35804-725f-43c7-857d-8459bbeb6b3b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:04.767897 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b6f35804-725f-43c7-857d-8459bbeb6b3b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:04.812815 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b6f35804-725f-43c7-857d-8459bbeb6b3b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:04.813666 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b6f35804-725f-43c7-857d-8459bbeb6b3b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:04.814307 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b6f35804-725f-43c7-857d-8459bbeb6b3b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:04.814827 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.objects.instance [None req-b6f35804-725f-43c7-857d-8459bbeb6b3b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lazy-loading 'fault' on Instance uuid db9ac414-31a5-4570-aa3f-37db3dac2049 {{(pid=118962) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 30 01:02:04.854825 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-4cfdee28-02d1-4371-8eaf-c450fb9e4201 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:04.864703 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-4cfdee28-02d1-4371-8eaf-c450fb9e4201 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:04.866166 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-4cfdee28-02d1-4371-8eaf-c450fb9e4201 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:04.867783 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-4cfdee28-02d1-4371-8eaf-c450fb9e4201 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1808 microversion: 2.20 time: 0.192594 Oct 30 01:02:04.868531 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 387/772] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:02:04 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1808 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:02:04.887428 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-b6f35804-725f-43c7-857d-8459bbeb6b3b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:04.895407 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-b6f35804-725f-43c7-857d-8459bbeb6b3b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049" status: 200 len: 1309 microversion: 2.1 time: 0.141444 Oct 30 01:02:04.896010 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 386/773] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:02:04 2022] GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049 => generated 1309 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:02:05.896903 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-c83e743e-664e-4146-8f82-31d3f2f6791f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:02:05.897467 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-c83e743e-664e-4146-8f82-31d3f2f6791f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:02:05.904388 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c83e743e-664e-4146-8f82-31d3f2f6791f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:05.904874 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c83e743e-664e-4146-8f82-31d3f2f6791f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:05.905346 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c83e743e-664e-4146-8f82-31d3f2f6791f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:05.906036 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c83e743e-664e-4146-8f82-31d3f2f6791f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:05.906497 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c83e743e-664e-4146-8f82-31d3f2f6791f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:05.906918 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c83e743e-664e-4146-8f82-31d3f2f6791f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:05.918822 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-631391ef-f20d-428b-933a-17f7c534671b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:02:05.919436 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-631391ef-f20d-428b-933a-17f7c534671b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: db9ac414-31a5-4570-aa3f-37db3dac2049] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:02:05.926187 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-631391ef-f20d-428b-933a-17f7c534671b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:05.926673 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-631391ef-f20d-428b-933a-17f7c534671b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:05.927131 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-631391ef-f20d-428b-933a-17f7c534671b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:05.927907 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-631391ef-f20d-428b-933a-17f7c534671b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:05.928636 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-631391ef-f20d-428b-933a-17f7c534671b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:05.929178 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-631391ef-f20d-428b-933a-17f7c534671b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:05.961624 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-631391ef-f20d-428b-933a-17f7c534671b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:05.962380 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-631391ef-f20d-428b-933a-17f7c534671b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:05.962807 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-631391ef-f20d-428b-933a-17f7c534671b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:05.963448 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.objects.instance [None req-631391ef-f20d-428b-933a-17f7c534671b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lazy-loading 'fault' on Instance uuid db9ac414-31a5-4570-aa3f-37db3dac2049 {{(pid=118962) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 30 01:02:06.015616 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-631391ef-f20d-428b-933a-17f7c534671b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:06.026446 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-631391ef-f20d-428b-933a-17f7c534671b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049" status: 200 len: 1309 microversion: 2.1 time: 0.111646 Oct 30 01:02:06.027335 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 387/774] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:02:05 2022] GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049 => generated 1309 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:02:06.074107 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-c83e743e-664e-4146-8f82-31d3f2f6791f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:06.083009 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-c83e743e-664e-4146-8f82-31d3f2f6791f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:06.084464 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-c83e743e-664e-4146-8f82-31d3f2f6791f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:06.085755 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-c83e743e-664e-4146-8f82-31d3f2f6791f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1808 microversion: 2.20 time: 0.194904 Oct 30 01:02:06.086304 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 388/775] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:02:05 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1808 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:02:07.054423 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-59220a3b-ac07-4de7-bcf8-390a76a8afde tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:02:07.055447 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-59220a3b-ac07-4de7-bcf8-390a76a8afde tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: db9ac414-31a5-4570-aa3f-37db3dac2049] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:02:07.063055 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-59220a3b-ac07-4de7-bcf8-390a76a8afde tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:07.063583 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-59220a3b-ac07-4de7-bcf8-390a76a8afde tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:07.064084 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-59220a3b-ac07-4de7-bcf8-390a76a8afde tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:07.064844 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-59220a3b-ac07-4de7-bcf8-390a76a8afde tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:07.065333 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-59220a3b-ac07-4de7-bcf8-390a76a8afde tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:07.065790 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-59220a3b-ac07-4de7-bcf8-390a76a8afde tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:07.099192 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-59220a3b-ac07-4de7-bcf8-390a76a8afde tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:07.099658 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-59220a3b-ac07-4de7-bcf8-390a76a8afde tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:07.100110 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-59220a3b-ac07-4de7-bcf8-390a76a8afde tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:07.100538 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.objects.instance [None req-59220a3b-ac07-4de7-bcf8-390a76a8afde tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lazy-loading 'fault' on Instance uuid db9ac414-31a5-4570-aa3f-37db3dac2049 {{(pid=118962) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 30 01:02:07.110623 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-b04a676d-1ad4-4b1d-92a5-45898197dd33 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:02:07.111288 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-b04a676d-1ad4-4b1d-92a5-45898197dd33 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:02:07.118304 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b04a676d-1ad4-4b1d-92a5-45898197dd33 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:07.118787 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b04a676d-1ad4-4b1d-92a5-45898197dd33 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:07.119213 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b04a676d-1ad4-4b1d-92a5-45898197dd33 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:07.119941 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b04a676d-1ad4-4b1d-92a5-45898197dd33 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:07.120373 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b04a676d-1ad4-4b1d-92a5-45898197dd33 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:07.120794 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b04a676d-1ad4-4b1d-92a5-45898197dd33 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:07.146851 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-59220a3b-ac07-4de7-bcf8-390a76a8afde tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:07.155460 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-59220a3b-ac07-4de7-bcf8-390a76a8afde tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049" status: 200 len: 1309 microversion: 2.1 time: 0.107524 Oct 30 01:02:07.156078 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 388/776] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:02:07 2022] GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049 => generated 1309 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:02:07.254970 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-b04a676d-1ad4-4b1d-92a5-45898197dd33 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:07.269307 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-b04a676d-1ad4-4b1d-92a5-45898197dd33 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:07.271488 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-b04a676d-1ad4-4b1d-92a5-45898197dd33 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:07.273385 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-b04a676d-1ad4-4b1d-92a5-45898197dd33 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1808 microversion: 2.20 time: 0.166320 Oct 30 01:02:07.274133 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 389/777] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:02:07 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1808 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:02:08.180005 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-fdb8d92a-d3eb-4207-aece-5b610495a6ce tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:02:08.180662 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-fdb8d92a-d3eb-4207-aece-5b610495a6ce tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: db9ac414-31a5-4570-aa3f-37db3dac2049] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:02:08.188483 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-fdb8d92a-d3eb-4207-aece-5b610495a6ce tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:08.189050 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-fdb8d92a-d3eb-4207-aece-5b610495a6ce tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:08.189557 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-fdb8d92a-d3eb-4207-aece-5b610495a6ce tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:08.190362 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-fdb8d92a-d3eb-4207-aece-5b610495a6ce tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:08.190866 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-fdb8d92a-d3eb-4207-aece-5b610495a6ce tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:08.191463 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-fdb8d92a-d3eb-4207-aece-5b610495a6ce tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:08.224377 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-fdb8d92a-d3eb-4207-aece-5b610495a6ce tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:08.224840 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-fdb8d92a-d3eb-4207-aece-5b610495a6ce tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:08.225272 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-fdb8d92a-d3eb-4207-aece-5b610495a6ce tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:08.225703 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.objects.instance [None req-fdb8d92a-d3eb-4207-aece-5b610495a6ce tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lazy-loading 'fault' on Instance uuid db9ac414-31a5-4570-aa3f-37db3dac2049 {{(pid=118962) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 30 01:02:08.267915 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-fdb8d92a-d3eb-4207-aece-5b610495a6ce tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:08.279655 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-fdb8d92a-d3eb-4207-aece-5b610495a6ce tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049" status: 200 len: 1309 microversion: 2.1 time: 0.104199 Oct 30 01:02:08.280525 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 389/778] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:02:08 2022] GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049 => generated 1309 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:02:08.299009 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-29844c55-adaa-4948-8ccf-4c63bc310395 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:02:08.299762 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-29844c55-adaa-4948-8ccf-4c63bc310395 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:02:08.306423 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-29844c55-adaa-4948-8ccf-4c63bc310395 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:08.306909 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-29844c55-adaa-4948-8ccf-4c63bc310395 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:08.307336 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-29844c55-adaa-4948-8ccf-4c63bc310395 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:08.308065 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-29844c55-adaa-4948-8ccf-4c63bc310395 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:08.308516 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-29844c55-adaa-4948-8ccf-4c63bc310395 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:08.308930 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-29844c55-adaa-4948-8ccf-4c63bc310395 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:08.447145 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-29844c55-adaa-4948-8ccf-4c63bc310395 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:08.456797 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-29844c55-adaa-4948-8ccf-4c63bc310395 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:08.458234 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-29844c55-adaa-4948-8ccf-4c63bc310395 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:08.459406 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-29844c55-adaa-4948-8ccf-4c63bc310395 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1808 microversion: 2.20 time: 0.164713 Oct 30 01:02:08.460006 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 390/779] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:02:08 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1808 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:02:09.307101 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-bfd3e955-baa3-42ad-9df7-1ae56eec841b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:02:09.307835 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-bfd3e955-baa3-42ad-9df7-1ae56eec841b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: db9ac414-31a5-4570-aa3f-37db3dac2049] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:02:09.316145 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bfd3e955-baa3-42ad-9df7-1ae56eec841b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:09.316728 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bfd3e955-baa3-42ad-9df7-1ae56eec841b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:09.317267 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bfd3e955-baa3-42ad-9df7-1ae56eec841b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:09.318096 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bfd3e955-baa3-42ad-9df7-1ae56eec841b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:09.318631 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bfd3e955-baa3-42ad-9df7-1ae56eec841b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:09.319118 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bfd3e955-baa3-42ad-9df7-1ae56eec841b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:09.355240 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bfd3e955-baa3-42ad-9df7-1ae56eec841b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:09.355746 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bfd3e955-baa3-42ad-9df7-1ae56eec841b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:09.356191 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bfd3e955-baa3-42ad-9df7-1ae56eec841b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:09.356631 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.objects.instance [None req-bfd3e955-baa3-42ad-9df7-1ae56eec841b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lazy-loading 'fault' on Instance uuid db9ac414-31a5-4570-aa3f-37db3dac2049 {{(pid=118962) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 30 01:02:09.400687 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-bfd3e955-baa3-42ad-9df7-1ae56eec841b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:09.412601 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-bfd3e955-baa3-42ad-9df7-1ae56eec841b tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049" status: 200 len: 1309 microversion: 2.1 time: 0.111075 Oct 30 01:02:09.413515 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 390/780] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:02:09 2022] GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049 => generated 1309 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:02:09.488623 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-57035b52-1ab1-4e5b-bbda-29b610496f28 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:02:09.489244 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-57035b52-1ab1-4e5b-bbda-29b610496f28 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:02:09.496261 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-57035b52-1ab1-4e5b-bbda-29b610496f28 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:09.496792 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-57035b52-1ab1-4e5b-bbda-29b610496f28 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:09.497227 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-57035b52-1ab1-4e5b-bbda-29b610496f28 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:09.497916 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-57035b52-1ab1-4e5b-bbda-29b610496f28 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:09.498365 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-57035b52-1ab1-4e5b-bbda-29b610496f28 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:09.498785 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-57035b52-1ab1-4e5b-bbda-29b610496f28 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:09.638219 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-57035b52-1ab1-4e5b-bbda-29b610496f28 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:09.651375 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-57035b52-1ab1-4e5b-bbda-29b610496f28 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:09.653658 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-57035b52-1ab1-4e5b-bbda-29b610496f28 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:09.655393 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-57035b52-1ab1-4e5b-bbda-29b610496f28 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1808 microversion: 2.20 time: 0.172556 Oct 30 01:02:09.656247 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 391/781] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:02:09 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1808 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:02:10.442723 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-33a137e3-6c06-484f-951c-0f2e462e2d94 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:02:10.443335 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-33a137e3-6c06-484f-951c-0f2e462e2d94 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: db9ac414-31a5-4570-aa3f-37db3dac2049] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:02:10.450489 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-33a137e3-6c06-484f-951c-0f2e462e2d94 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:10.450949 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-33a137e3-6c06-484f-951c-0f2e462e2d94 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:10.451372 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-33a137e3-6c06-484f-951c-0f2e462e2d94 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:10.452090 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-33a137e3-6c06-484f-951c-0f2e462e2d94 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:10.452529 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-33a137e3-6c06-484f-951c-0f2e462e2d94 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:10.452937 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-33a137e3-6c06-484f-951c-0f2e462e2d94 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:10.484660 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-33a137e3-6c06-484f-951c-0f2e462e2d94 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:10.485126 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-33a137e3-6c06-484f-951c-0f2e462e2d94 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:10.485608 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-33a137e3-6c06-484f-951c-0f2e462e2d94 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:10.486057 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.objects.instance [None req-33a137e3-6c06-484f-951c-0f2e462e2d94 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lazy-loading 'fault' on Instance uuid db9ac414-31a5-4570-aa3f-37db3dac2049 {{(pid=118962) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 30 01:02:10.535117 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-33a137e3-6c06-484f-951c-0f2e462e2d94 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:10.543243 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-33a137e3-6c06-484f-951c-0f2e462e2d94 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049" status: 200 len: 1309 microversion: 2.1 time: 0.104472 Oct 30 01:02:10.543811 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 391/782] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:02:10 2022] GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049 => generated 1309 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:02:10.680964 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-b431f2dc-6ec2-4941-a2b2-c836e95b468f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:02:10.681687 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-b431f2dc-6ec2-4941-a2b2-c836e95b468f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:02:10.688534 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b431f2dc-6ec2-4941-a2b2-c836e95b468f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:10.689016 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b431f2dc-6ec2-4941-a2b2-c836e95b468f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:10.689598 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b431f2dc-6ec2-4941-a2b2-c836e95b468f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:10.690293 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b431f2dc-6ec2-4941-a2b2-c836e95b468f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:10.690717 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b431f2dc-6ec2-4941-a2b2-c836e95b468f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:10.691158 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b431f2dc-6ec2-4941-a2b2-c836e95b468f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:10.831193 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-b431f2dc-6ec2-4941-a2b2-c836e95b468f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:10.840504 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-b431f2dc-6ec2-4941-a2b2-c836e95b468f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:10.841932 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-b431f2dc-6ec2-4941-a2b2-c836e95b468f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:10.843156 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-b431f2dc-6ec2-4941-a2b2-c836e95b468f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1808 microversion: 2.20 time: 0.166227 Oct 30 01:02:10.843751 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 392/783] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:02:10 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1808 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:02:11.571662 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-0781850d-8920-4eb3-a744-9b2876e9e21d tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:02:11.572627 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-0781850d-8920-4eb3-a744-9b2876e9e21d tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: db9ac414-31a5-4570-aa3f-37db3dac2049] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:02:11.583773 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0781850d-8920-4eb3-a744-9b2876e9e21d tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:11.584602 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0781850d-8920-4eb3-a744-9b2876e9e21d tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:11.585307 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0781850d-8920-4eb3-a744-9b2876e9e21d tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:11.586422 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0781850d-8920-4eb3-a744-9b2876e9e21d tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:11.587208 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0781850d-8920-4eb3-a744-9b2876e9e21d tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:11.587969 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0781850d-8920-4eb3-a744-9b2876e9e21d tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:11.619144 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0781850d-8920-4eb3-a744-9b2876e9e21d tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:11.619873 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0781850d-8920-4eb3-a744-9b2876e9e21d tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:11.620312 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0781850d-8920-4eb3-a744-9b2876e9e21d tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:11.620709 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.objects.instance [None req-0781850d-8920-4eb3-a744-9b2876e9e21d tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lazy-loading 'fault' on Instance uuid db9ac414-31a5-4570-aa3f-37db3dac2049 {{(pid=118962) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 30 01:02:11.661906 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-0781850d-8920-4eb3-a744-9b2876e9e21d tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddeaa7a90ee7483590e6bc0d189e1e2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2fbc73a853584aaba6109090bd6b1281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:11.669967 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-0781850d-8920-4eb3-a744-9b2876e9e21d tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049" status: 200 len: 1309 microversion: 2.1 time: 0.104877 Oct 30 01:02:11.670540 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 392/784] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:02:11 2022] GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049 => generated 1309 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:02:11.881226 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-09173e04-7118-43f8-a978-1e0d969540cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:02:11.882073 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-09173e04-7118-43f8-a978-1e0d969540cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:02:11.890610 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-09173e04-7118-43f8-a978-1e0d969540cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:11.891096 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-09173e04-7118-43f8-a978-1e0d969540cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:11.891528 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-09173e04-7118-43f8-a978-1e0d969540cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:11.892259 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-09173e04-7118-43f8-a978-1e0d969540cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:11.892711 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-09173e04-7118-43f8-a978-1e0d969540cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:11.893129 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-09173e04-7118-43f8-a978-1e0d969540cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:12.033201 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-09173e04-7118-43f8-a978-1e0d969540cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:12.042092 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-09173e04-7118-43f8-a978-1e0d969540cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:12.043549 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-09173e04-7118-43f8-a978-1e0d969540cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:12.044779 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-09173e04-7118-43f8-a978-1e0d969540cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1808 microversion: 2.20 time: 0.170370 Oct 30 01:02:12.045355 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 393/785] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:02:11 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1808 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:02:12.693225 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-95a4a192-55d9-4804-b9b4-951b2d7b1a32 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:02:12.693841 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-95a4a192-55d9-4804-b9b4-951b2d7b1a32 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] [instance: db9ac414-31a5-4570-aa3f-37db3dac2049] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:02:12.700682 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-95a4a192-55d9-4804-b9b4-951b2d7b1a32 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:12.701157 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-95a4a192-55d9-4804-b9b4-951b2d7b1a32 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:12.701578 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-95a4a192-55d9-4804-b9b4-951b2d7b1a32 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:12.702227 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-95a4a192-55d9-4804-b9b4-951b2d7b1a32 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:12.702669 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-95a4a192-55d9-4804-b9b4-951b2d7b1a32 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:12.703166 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-95a4a192-55d9-4804-b9b4-951b2d7b1a32 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:12.717692 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.wsgi [None req-95a4a192-55d9-4804-b9b4-951b2d7b1a32 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] HTTP exception thrown: Instance db9ac414-31a5-4570-aa3f-37db3dac2049 could not be found. Oct 30 01:02:12.719407 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-95a4a192-55d9-4804-b9b4-951b2d7b1a32 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Request method failure captured: Oct 30 01:02:12.719407 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: request: GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049 HTTP/1.1 Oct 30 01:02:12.719407 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept: application/json Oct 30 01:02:12.719407 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept-Encoding: identity Oct 30 01:02:12.719407 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Connection: close Oct 30 01:02:12.719407 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Length: 0 Oct 30 01:02:12.719407 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Type: application/json Oct 30 01:02:12.719407 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Host: 10.222.0.16 Oct 30 01:02:12.719407 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Openstack-System-Scope: None Oct 30 01:02:12.719407 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: User-Agent: python-urllib3/1.26.12 Oct 30 01:02:12.719407 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Auth-Token: gAAAAABjXcwpEK4P3cM0ywCLZB1a-Aa4hWbP8jTJ4dGTi7fm1Gw71o6nFRneJTYcyExGVypGWFJMASQBgQOkcdgBwyyFxBmsEpyudU9H41QTjw7SzfiyEzKMQ4OwemWWuMPbeXo3pfyEMR9xuKPgDSRAEQCefZCab_ZKaD658JjLDxFMQOGtrcs Oct 30 01:02:12.719407 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Id: None Oct 30 01:02:12.719407 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Name: None Oct 30 01:02:12.719407 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Identity-Status: Confirmed Oct 30 01:02:12.719407 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Is-Admin-Project: True Oct 30 01:02:12.719407 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Id: default Oct 30 01:02:12.719407 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Name: Default Oct 30 01:02:12.719407 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Id: 2fbc73a853584aaba6109090bd6b1281 Oct 30 01:02:12.719407 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Name: tempest-VolumesSnapshotTestJSON-761222885 Oct 30 01:02:12.719407 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Role: reader,member Oct 30 01:02:12.719407 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Roles: reader,member Oct 30 01:02:12.720823 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_2fbc73a853584aaba6109090bd6b1281"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2fbc73a853584aaba6109090bd6b1281"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/2fbc73a853584aaba6109090bd6b1281"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2fbc73a853584aaba6109090bd6b1281"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 01:02:12.720823 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant: tempest-VolumesSnapshotTestJSON-761222885 Oct 30 01:02:12.720823 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Id: 2fbc73a853584aaba6109090bd6b1281 Oct 30 01:02:12.720823 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-761222885 Oct 30 01:02:12.720823 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User: tempest-VolumesSnapshotTestJSON-761222885-project Oct 30 01:02:12.720823 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Id: default Oct 30 01:02:12.720823 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Name: Default Oct 30 01:02:12.720823 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Id: ddeaa7a90ee7483590e6bc0d189e1e2a Oct 30 01:02:12.720823 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Name: tempest-VolumesSnapshotTestJSON-761222885-project Oct 30 01:02:12.720823 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: method: > Oct 30 01:02:12.720823 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: exception: Instance db9ac414-31a5-4570-aa3f-37db3dac2049 could not be found. Oct 30 01:02:12.720823 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: action_args: {'id': 'db9ac414-31a5-4570-aa3f-37db3dac2049'} Oct 30 01:02:12.721572 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 01:02:12.721572 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:02:12.721572 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 30 01:02:12.721572 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 30 01:02:12.721572 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 30 01:02:12.721572 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 30 01:02:12.721572 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 30 01:02:12.721572 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 30 01:02:12.721572 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 30 01:02:12.721572 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 30 01:02:12.721572 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance db9ac414-31a5-4570-aa3f-37db3dac2049 could not be found. Oct 30 01:02:12.721572 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:02:12.721572 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:02:12.721572 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:02:12.721572 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:02:12.721572 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:02:12.721572 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:02:12.721572 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 01:02:12.721572 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 01:02:12.721572 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 01:02:12.722807 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 01:02:12.722807 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 30 01:02:12.722807 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 30 01:02:12.722807 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 30 01:02:12.722807 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 30 01:02:12.722807 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 30 01:02:12.722807 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 30 01:02:12.722807 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance db9ac414-31a5-4570-aa3f-37db3dac2049 could not be found. Oct 30 01:02:12.722807 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:02:12.722807 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:02:12.722807 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:02:12.722807 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:02:12.722807 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:02:12.722807 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:02:12.722807 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 01:02:12.722807 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 01:02:12.722807 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance db9ac414-31a5-4570-aa3f-37db3dac2049 could not be found. Oct 30 01:02:12.722807 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:02:12.723927 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-95a4a192-55d9-4804-b9b4-951b2d7b1a32 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] Returning 404 to user: Instance db9ac414-31a5-4570-aa3f-37db3dac2049 could not be found. {{(pid=118962) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 01:02:12.723927 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-95a4a192-55d9-4804-b9b4-951b2d7b1a32 tempest-VolumesSnapshotTestJSON-761222885 tempest-VolumesSnapshotTestJSON-761222885-project] 10.222.0.16 "GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049" status: 404 len: 111 microversion: 2.1 time: 0.031723 Oct 30 01:02:12.723927 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 393/786] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:02:12 2022] GET /compute/v2.1/servers/db9ac414-31a5-4570-aa3f-37db3dac2049 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 30 01:02:13.070406 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-250d0162-5301-4084-8919-4fdbfe9826b5 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:02:13.070997 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-250d0162-5301-4084-8919-4fdbfe9826b5 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:02:13.078037 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-250d0162-5301-4084-8919-4fdbfe9826b5 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:13.078520 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-250d0162-5301-4084-8919-4fdbfe9826b5 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:13.078933 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-250d0162-5301-4084-8919-4fdbfe9826b5 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:13.079640 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-250d0162-5301-4084-8919-4fdbfe9826b5 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:13.080450 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-250d0162-5301-4084-8919-4fdbfe9826b5 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:13.080869 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-250d0162-5301-4084-8919-4fdbfe9826b5 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:13.217080 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-250d0162-5301-4084-8919-4fdbfe9826b5 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:13.226581 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-250d0162-5301-4084-8919-4fdbfe9826b5 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:13.228028 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-250d0162-5301-4084-8919-4fdbfe9826b5 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:13.229185 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-250d0162-5301-4084-8919-4fdbfe9826b5 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1808 microversion: 2.20 time: 0.162777 Oct 30 01:02:13.229747 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 394/787] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:02:13 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1808 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:02:14.256112 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-4d884deb-f8f0-450e-9f14-3c07e1ebbe2d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:02:14.256678 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-4d884deb-f8f0-450e-9f14-3c07e1ebbe2d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:02:14.270016 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4d884deb-f8f0-450e-9f14-3c07e1ebbe2d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:14.270502 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4d884deb-f8f0-450e-9f14-3c07e1ebbe2d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:14.270994 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4d884deb-f8f0-450e-9f14-3c07e1ebbe2d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:14.271636 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4d884deb-f8f0-450e-9f14-3c07e1ebbe2d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:14.272116 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4d884deb-f8f0-450e-9f14-3c07e1ebbe2d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:14.272530 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4d884deb-f8f0-450e-9f14-3c07e1ebbe2d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:14.410788 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-4d884deb-f8f0-450e-9f14-3c07e1ebbe2d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:14.419572 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-4d884deb-f8f0-450e-9f14-3c07e1ebbe2d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:14.421064 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-4d884deb-f8f0-450e-9f14-3c07e1ebbe2d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:14.422222 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-4d884deb-f8f0-450e-9f14-3c07e1ebbe2d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1808 microversion: 2.20 time: 0.171541 Oct 30 01:02:14.422748 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 394/788] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:02:14 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1808 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:02:15.452073 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-c01801c6-052d-4b5b-9e54-5ec687e92cb3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:02:15.453131 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-c01801c6-052d-4b5b-9e54-5ec687e92cb3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:02:15.464647 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c01801c6-052d-4b5b-9e54-5ec687e92cb3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:15.465417 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c01801c6-052d-4b5b-9e54-5ec687e92cb3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:15.465856 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c01801c6-052d-4b5b-9e54-5ec687e92cb3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:15.466551 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c01801c6-052d-4b5b-9e54-5ec687e92cb3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:15.467003 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c01801c6-052d-4b5b-9e54-5ec687e92cb3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:15.467423 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c01801c6-052d-4b5b-9e54-5ec687e92cb3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:15.605546 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-c01801c6-052d-4b5b-9e54-5ec687e92cb3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:15.614340 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-c01801c6-052d-4b5b-9e54-5ec687e92cb3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:15.615748 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-c01801c6-052d-4b5b-9e54-5ec687e92cb3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:15.616942 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-c01801c6-052d-4b5b-9e54-5ec687e92cb3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1808 microversion: 2.20 time: 0.171790 Oct 30 01:02:15.617505 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 395/789] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:02:15 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1808 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:02:16.646863 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-2b40ac5a-b922-4ee9-b523-a4c12d5cabc9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:02:16.647751 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-2b40ac5a-b922-4ee9-b523-a4c12d5cabc9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:02:16.651445 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_api:SELECT=92 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:02:16.657693 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2b40ac5a-b922-4ee9-b523-a4c12d5cabc9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:16.658177 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2b40ac5a-b922-4ee9-b523-a4c12d5cabc9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:16.658604 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2b40ac5a-b922-4ee9-b523-a4c12d5cabc9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:16.659291 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2b40ac5a-b922-4ee9-b523-a4c12d5cabc9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:16.659755 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2b40ac5a-b922-4ee9-b523-a4c12d5cabc9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:16.660191 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2b40ac5a-b922-4ee9-b523-a4c12d5cabc9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:16.679431 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell1:SELECT=100 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:02:16.792694 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-2b40ac5a-b922-4ee9-b523-a4c12d5cabc9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:16.801819 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-2b40ac5a-b922-4ee9-b523-a4c12d5cabc9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:16.803277 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-2b40ac5a-b922-4ee9-b523-a4c12d5cabc9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:16.804485 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-2b40ac5a-b922-4ee9-b523-a4c12d5cabc9 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1808 microversion: 2.20 time: 0.164474 Oct 30 01:02:16.805113 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 395/790] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:02:16 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1808 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:02:17.833492 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-8c4e8b7b-0087-413d-b1ba-8a05d45833ae tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:02:17.834463 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-8c4e8b7b-0087-413d-b1ba-8a05d45833ae tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:02:17.844709 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8c4e8b7b-0087-413d-b1ba-8a05d45833ae tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:17.845493 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8c4e8b7b-0087-413d-b1ba-8a05d45833ae tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:17.846160 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8c4e8b7b-0087-413d-b1ba-8a05d45833ae tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:17.847218 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8c4e8b7b-0087-413d-b1ba-8a05d45833ae tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:17.847957 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8c4e8b7b-0087-413d-b1ba-8a05d45833ae tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:17.848611 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8c4e8b7b-0087-413d-b1ba-8a05d45833ae tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:17.988630 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-8c4e8b7b-0087-413d-b1ba-8a05d45833ae tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:18.002034 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-8c4e8b7b-0087-413d-b1ba-8a05d45833ae tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:18.004373 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-8c4e8b7b-0087-413d-b1ba-8a05d45833ae tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:18.006128 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-8c4e8b7b-0087-413d-b1ba-8a05d45833ae tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1808 microversion: 2.20 time: 0.179039 Oct 30 01:02:18.007033 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 396/791] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:02:17 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1808 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:02:19.036115 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-56bbe7bf-1ba6-47b6-9a59-34d0a3bfa05e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:02:19.036863 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-56bbe7bf-1ba6-47b6-9a59-34d0a3bfa05e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:02:19.045396 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-56bbe7bf-1ba6-47b6-9a59-34d0a3bfa05e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:19.045994 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-56bbe7bf-1ba6-47b6-9a59-34d0a3bfa05e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:19.046531 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-56bbe7bf-1ba6-47b6-9a59-34d0a3bfa05e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:19.047387 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-56bbe7bf-1ba6-47b6-9a59-34d0a3bfa05e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:19.047975 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-56bbe7bf-1ba6-47b6-9a59-34d0a3bfa05e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:19.048501 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-56bbe7bf-1ba6-47b6-9a59-34d0a3bfa05e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:19.197046 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-56bbe7bf-1ba6-47b6-9a59-34d0a3bfa05e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:19.207104 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-56bbe7bf-1ba6-47b6-9a59-34d0a3bfa05e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:19.208694 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-56bbe7bf-1ba6-47b6-9a59-34d0a3bfa05e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:19.209862 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-56bbe7bf-1ba6-47b6-9a59-34d0a3bfa05e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1808 microversion: 2.20 time: 0.178837 Oct 30 01:02:19.210430 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 396/792] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:02:19 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1808 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:02:20.240650 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-63434629-ad8f-4e97-8a5e-da5f942321e6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:02:20.241520 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-63434629-ad8f-4e97-8a5e-da5f942321e6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:02:20.252627 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-63434629-ad8f-4e97-8a5e-da5f942321e6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:20.253419 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-63434629-ad8f-4e97-8a5e-da5f942321e6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:20.254089 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-63434629-ad8f-4e97-8a5e-da5f942321e6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:20.255224 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-63434629-ad8f-4e97-8a5e-da5f942321e6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:20.256039 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-63434629-ad8f-4e97-8a5e-da5f942321e6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:20.256738 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-63434629-ad8f-4e97-8a5e-da5f942321e6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:20.395780 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-63434629-ad8f-4e97-8a5e-da5f942321e6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:20.399291 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell1:SELECT=100 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:02:20.405735 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-63434629-ad8f-4e97-8a5e-da5f942321e6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:20.407180 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-63434629-ad8f-4e97-8a5e-da5f942321e6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:20.408391 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-63434629-ad8f-4e97-8a5e-da5f942321e6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1808 microversion: 2.20 time: 0.174720 Oct 30 01:02:20.408939 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 397/793] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:02:20 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1808 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:02:21.434598 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-84c6123d-7c4b-4119-a8bc-4c025311b9e6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:02:21.435188 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-84c6123d-7c4b-4119-a8bc-4c025311b9e6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:02:21.441694 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-84c6123d-7c4b-4119-a8bc-4c025311b9e6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:21.442176 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-84c6123d-7c4b-4119-a8bc-4c025311b9e6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:21.442600 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-84c6123d-7c4b-4119-a8bc-4c025311b9e6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:21.443306 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-84c6123d-7c4b-4119-a8bc-4c025311b9e6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:21.443785 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-84c6123d-7c4b-4119-a8bc-4c025311b9e6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:21.444206 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-84c6123d-7c4b-4119-a8bc-4c025311b9e6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:21.578833 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-84c6123d-7c4b-4119-a8bc-4c025311b9e6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:21.588246 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-84c6123d-7c4b-4119-a8bc-4c025311b9e6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:21.589666 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-84c6123d-7c4b-4119-a8bc-4c025311b9e6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:21.590822 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-84c6123d-7c4b-4119-a8bc-4c025311b9e6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1808 microversion: 2.20 time: 0.160238 Oct 30 01:02:21.591376 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 397/794] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:02:21 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1808 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:02:22.621422 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-1e081a5b-1225-4105-86e4-45c3ea715640 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:02:22.622584 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-1e081a5b-1225-4105-86e4-45c3ea715640 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:02:22.625887 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_api:SELECT=48 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:02:22.631349 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1e081a5b-1225-4105-86e4-45c3ea715640 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:22.631856 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1e081a5b-1225-4105-86e4-45c3ea715640 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:22.632289 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1e081a5b-1225-4105-86e4-45c3ea715640 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:22.633106 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1e081a5b-1225-4105-86e4-45c3ea715640 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:22.633553 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1e081a5b-1225-4105-86e4-45c3ea715640 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:22.633977 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1e081a5b-1225-4105-86e4-45c3ea715640 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:22.765261 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-1e081a5b-1225-4105-86e4-45c3ea715640 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:22.774658 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-1e081a5b-1225-4105-86e4-45c3ea715640 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:22.776094 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-1e081a5b-1225-4105-86e4-45c3ea715640 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:22.777266 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-1e081a5b-1225-4105-86e4-45c3ea715640 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1808 microversion: 2.20 time: 0.162955 Oct 30 01:02:22.777800 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 398/795] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:02:22 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1808 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:02:23.036289 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [req-38c63d18-ee4c-4fa6-a602-7794ea235fb0 req-431f1e76-af92-4097-aa2e-87ef1a38622a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9491a8f4-2b76-4a4e-a6bd-6720114e482f", "name": "network-vif-plugged", "status": "completed", "tag": "5367850e-73d0-4814-af96-bd430c94c272"}]} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:02:23.046061 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-38c63d18-ee4c-4fa6-a602-7794ea235fb0 req-431f1e76-af92-4097-aa2e-87ef1a38622a service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:23.046624 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-38c63d18-ee4c-4fa6-a602-7794ea235fb0 req-431f1e76-af92-4097-aa2e-87ef1a38622a service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:23.047155 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-38c63d18-ee4c-4fa6-a602-7794ea235fb0 req-431f1e76-af92-4097-aa2e-87ef1a38622a service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:23.058545 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.compute.server_external_events [req-38c63d18-ee4c-4fa6-a602-7794ea235fb0 req-431f1e76-af92-4097-aa2e-87ef1a38622a service nova] Creating event network-vif-plugged:5367850e-73d0-4814-af96-bd430c94c272 for instance 9491a8f4-2b76-4a4e-a6bd-6720114e482f on hv-cinder-maste Oct 30 01:02:23.064241 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [req-38c63d18-ee4c-4fa6-a602-7794ea235fb0 req-431f1e76-af92-4097-aa2e-87ef1a38622a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.220904 Oct 30 01:02:23.064799 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 398/796] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:02:22 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:02:23.803142 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-7027ac59-b05e-44b9-bfd1-165390a5414e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:02:23.803735 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-7027ac59-b05e-44b9-bfd1-165390a5414e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:02:23.810520 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7027ac59-b05e-44b9-bfd1-165390a5414e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:23.811012 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7027ac59-b05e-44b9-bfd1-165390a5414e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:23.811446 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7027ac59-b05e-44b9-bfd1-165390a5414e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:23.812175 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7027ac59-b05e-44b9-bfd1-165390a5414e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:23.812619 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7027ac59-b05e-44b9-bfd1-165390a5414e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:23.813034 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7027ac59-b05e-44b9-bfd1-165390a5414e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:23.952581 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-7027ac59-b05e-44b9-bfd1-165390a5414e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:23.961949 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-7027ac59-b05e-44b9-bfd1-165390a5414e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:23.963369 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-7027ac59-b05e-44b9-bfd1-165390a5414e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:23.964554 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-7027ac59-b05e-44b9-bfd1-165390a5414e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1808 microversion: 2.20 time: 0.165737 Oct 30 01:02:23.965112 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 399/797] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:02:23 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1808 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:02:24.997354 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-554f711c-80b9-42f4-8595-4a567f1f9dde tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:02:24.998374 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-554f711c-80b9-42f4-8595-4a567f1f9dde tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:02:25.006384 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-554f711c-80b9-42f4-8595-4a567f1f9dde tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:25.006862 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-554f711c-80b9-42f4-8595-4a567f1f9dde tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:25.007290 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-554f711c-80b9-42f4-8595-4a567f1f9dde tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:25.008020 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-554f711c-80b9-42f4-8595-4a567f1f9dde tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:25.008466 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-554f711c-80b9-42f4-8595-4a567f1f9dde tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:25.008883 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-554f711c-80b9-42f4-8595-4a567f1f9dde tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:25.149200 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-554f711c-80b9-42f4-8595-4a567f1f9dde tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:25.158478 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-554f711c-80b9-42f4-8595-4a567f1f9dde tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:25.160036 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-554f711c-80b9-42f4-8595-4a567f1f9dde tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:25.161091 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-554f711c-80b9-42f4-8595-4a567f1f9dde tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1808 microversion: 2.20 time: 0.172051 Oct 30 01:02:25.161703 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 399/798] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:02:24 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1808 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:02:26.190579 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-e36896fb-9750-4767-96e0-218787fbdf10 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:02:26.191176 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-e36896fb-9750-4767-96e0-218787fbdf10 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:02:26.197709 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e36896fb-9750-4767-96e0-218787fbdf10 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:26.198194 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e36896fb-9750-4767-96e0-218787fbdf10 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:26.198631 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e36896fb-9750-4767-96e0-218787fbdf10 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:26.199321 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e36896fb-9750-4767-96e0-218787fbdf10 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:26.199798 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e36896fb-9750-4767-96e0-218787fbdf10 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:26.200221 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e36896fb-9750-4767-96e0-218787fbdf10 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:26.340720 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-e36896fb-9750-4767-96e0-218787fbdf10 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:26.350437 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-e36896fb-9750-4767-96e0-218787fbdf10 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:26.352703 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-e36896fb-9750-4767-96e0-218787fbdf10 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:26.354467 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-e36896fb-9750-4767-96e0-218787fbdf10 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1808 microversion: 2.20 time: 0.169900 Oct 30 01:02:26.355281 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 400/799] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:02:26 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1808 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:02:27.382046 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-e0de4bdb-7811-4ed1-a39a-2f75a440a3e3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:02:27.382681 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-e0de4bdb-7811-4ed1-a39a-2f75a440a3e3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:02:27.389931 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e0de4bdb-7811-4ed1-a39a-2f75a440a3e3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:27.390430 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e0de4bdb-7811-4ed1-a39a-2f75a440a3e3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:27.390853 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e0de4bdb-7811-4ed1-a39a-2f75a440a3e3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:27.391559 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e0de4bdb-7811-4ed1-a39a-2f75a440a3e3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:27.392097 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e0de4bdb-7811-4ed1-a39a-2f75a440a3e3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:27.392514 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e0de4bdb-7811-4ed1-a39a-2f75a440a3e3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:27.529691 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-e0de4bdb-7811-4ed1-a39a-2f75a440a3e3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:27.538777 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-e0de4bdb-7811-4ed1-a39a-2f75a440a3e3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:27.540236 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-e0de4bdb-7811-4ed1-a39a-2f75a440a3e3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:27.541391 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-e0de4bdb-7811-4ed1-a39a-2f75a440a3e3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1808 microversion: 2.20 time: 0.163487 Oct 30 01:02:27.541956 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 400/800] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:02:27 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1808 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:02:28.570892 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-b6691318-5143-4e03-8dd5-1d82a09acdba tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:02:28.571914 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-b6691318-5143-4e03-8dd5-1d82a09acdba tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:02:28.583292 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b6691318-5143-4e03-8dd5-1d82a09acdba tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:28.583827 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b6691318-5143-4e03-8dd5-1d82a09acdba tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:28.584259 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b6691318-5143-4e03-8dd5-1d82a09acdba tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:28.584949 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b6691318-5143-4e03-8dd5-1d82a09acdba tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:28.585395 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b6691318-5143-4e03-8dd5-1d82a09acdba tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:28.585808 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b6691318-5143-4e03-8dd5-1d82a09acdba tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:28.736138 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-b6691318-5143-4e03-8dd5-1d82a09acdba tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:28.745429 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-b6691318-5143-4e03-8dd5-1d82a09acdba tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:28.746848 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-b6691318-5143-4e03-8dd5-1d82a09acdba tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:28.748043 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-b6691318-5143-4e03-8dd5-1d82a09acdba tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1808 microversion: 2.20 time: 0.183999 Oct 30 01:02:28.748928 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 401/801] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:02:28 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1808 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:02:29.777019 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-08df9b25-97bd-4c76-9f23-7a7cb279d1f4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:02:29.778043 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-08df9b25-97bd-4c76-9f23-7a7cb279d1f4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:02:29.787444 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-08df9b25-97bd-4c76-9f23-7a7cb279d1f4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:29.787963 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-08df9b25-97bd-4c76-9f23-7a7cb279d1f4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:29.788394 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-08df9b25-97bd-4c76-9f23-7a7cb279d1f4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:29.789087 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-08df9b25-97bd-4c76-9f23-7a7cb279d1f4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:29.789530 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-08df9b25-97bd-4c76-9f23-7a7cb279d1f4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:29.789943 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-08df9b25-97bd-4c76-9f23-7a7cb279d1f4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:29.931071 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-08df9b25-97bd-4c76-9f23-7a7cb279d1f4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:29.939878 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-08df9b25-97bd-4c76-9f23-7a7cb279d1f4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:29.941293 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-08df9b25-97bd-4c76-9f23-7a7cb279d1f4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:29.942463 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-08df9b25-97bd-4c76-9f23-7a7cb279d1f4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1808 microversion: 2.20 time: 0.172325 Oct 30 01:02:29.943029 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 401/802] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:02:29 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1808 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:02:30.973903 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-9731fa0c-82af-4ec9-9f81-d1c88ffa5467 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:02:30.975439 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-9731fa0c-82af-4ec9-9f81-d1c88ffa5467 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:02:30.982814 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9731fa0c-82af-4ec9-9f81-d1c88ffa5467 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:30.983325 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9731fa0c-82af-4ec9-9f81-d1c88ffa5467 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:30.984037 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9731fa0c-82af-4ec9-9f81-d1c88ffa5467 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:30.984800 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9731fa0c-82af-4ec9-9f81-d1c88ffa5467 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:30.985504 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9731fa0c-82af-4ec9-9f81-d1c88ffa5467 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:30.986098 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9731fa0c-82af-4ec9-9f81-d1c88ffa5467 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:31.140877 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-9731fa0c-82af-4ec9-9f81-d1c88ffa5467 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:31.160170 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-9731fa0c-82af-4ec9-9f81-d1c88ffa5467 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:31.162180 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-9731fa0c-82af-4ec9-9f81-d1c88ffa5467 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:31.164045 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-9731fa0c-82af-4ec9-9f81-d1c88ffa5467 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1808 microversion: 2.20 time: 0.194062 Oct 30 01:02:31.176524 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 402/803] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:02:30 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1808 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:02:32.193990 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-0198e58e-e158-4c65-82b8-99a1cd0c039c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:02:32.194803 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-0198e58e-e158-4c65-82b8-99a1cd0c039c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:02:32.204251 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0198e58e-e158-4c65-82b8-99a1cd0c039c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:32.205021 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0198e58e-e158-4c65-82b8-99a1cd0c039c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:32.206103 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0198e58e-e158-4c65-82b8-99a1cd0c039c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:32.207217 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0198e58e-e158-4c65-82b8-99a1cd0c039c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:32.208181 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0198e58e-e158-4c65-82b8-99a1cd0c039c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:32.208818 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0198e58e-e158-4c65-82b8-99a1cd0c039c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:32.341323 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-0198e58e-e158-4c65-82b8-99a1cd0c039c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:32.350724 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-0198e58e-e158-4c65-82b8-99a1cd0c039c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:32.352179 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-0198e58e-e158-4c65-82b8-99a1cd0c039c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:32.353351 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-0198e58e-e158-4c65-82b8-99a1cd0c039c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1795 microversion: 2.20 time: 0.166805 Oct 30 01:02:32.353920 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 402/804] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:02:32 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1795 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:02:52.152248 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-220f5160-0ef7-4a8d-b8ab-a718ed8c0c5a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:02:52.153692 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-220f5160-0ef7-4a8d-b8ab-a718ed8c0c5a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:02:52.160281 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_api:SELECT=9 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:02:52.162222 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell1:SELECT=30 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:02:52.168372 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-220f5160-0ef7-4a8d-b8ab-a718ed8c0c5a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:52.169145 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-220f5160-0ef7-4a8d-b8ab-a718ed8c0c5a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:52.169809 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-220f5160-0ef7-4a8d-b8ab-a718ed8c0c5a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:52.171247 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-220f5160-0ef7-4a8d-b8ab-a718ed8c0c5a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:52.172093 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-220f5160-0ef7-4a8d-b8ab-a718ed8c0c5a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:52.172873 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-220f5160-0ef7-4a8d-b8ab-a718ed8c0c5a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:52.271228 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.objects.instance [None req-220f5160-0ef7-4a8d-b8ab-a718ed8c0c5a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lazy-loading 'services' on Instance uuid 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 {{(pid=118961) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 30 01:02:52.406840 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-220f5160-0ef7-4a8d-b8ab-a718ed8c0c5a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "DELETE /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14/os-volume_attachments/a9d9aa2e-2337-4976-9752-1910f19a8aec" status: 202 len: 0 microversion: 2.1 time: 0.258905 Oct 30 01:02:52.411246 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 403/805] 10.222.0.16 () {58 vars in 1416 bytes} [Sun Oct 30 01:02:52 2022] DELETE /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14/os-volume_attachments/a9d9aa2e-2337-4976-9752-1910f19a8aec => generated 0 bytes in 264 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 30 01:02:54.658676 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-e0aa705f-91a9-4ff3-afb2-2223e0aa9593 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:02:54.663030 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_api:SELECT=15 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:02:54.674718 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell1:SELECT=40 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:02:54.675681 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 30 01:02:54.675861 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Sun Oct 30 01:02:54 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14/action (ip 10.222.0.16) !!! Oct 30 01:02:54.685728 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e0aa705f-91a9-4ff3-afb2-2223e0aa9593 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:54.686218 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e0aa705f-91a9-4ff3-afb2-2223e0aa9593 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:54.686655 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e0aa705f-91a9-4ff3-afb2-2223e0aa9593 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:54.779581 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-e0aa705f-91a9-4ff3-afb2-2223e0aa9593 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Going to try to stop instance {{(pid=118962) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Oct 30 01:02:54.849185 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-e0aa705f-91a9-4ff3-afb2-2223e0aa9593 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "POST /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14/action" status: 202 len: 0 microversion: 2.1 time: 0.194255 Oct 30 01:02:54.850136 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 403/806] 10.222.0.16 () {60 vars in 1278 bytes} [Sun Oct 30 01:02:54 2022] POST /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14/action => generated 0 bytes in 196 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 30 01:02:54.860511 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-692616c7-07e3-4bfd-8cf9-55a47a3b209f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:02:54.861231 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-692616c7-07e3-4bfd-8cf9-55a47a3b209f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:02:54.868326 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-692616c7-07e3-4bfd-8cf9-55a47a3b209f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:54.869098 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-692616c7-07e3-4bfd-8cf9-55a47a3b209f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:54.869720 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-692616c7-07e3-4bfd-8cf9-55a47a3b209f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:54.870671 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-692616c7-07e3-4bfd-8cf9-55a47a3b209f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:54.871288 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-692616c7-07e3-4bfd-8cf9-55a47a3b209f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:54.871978 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-692616c7-07e3-4bfd-8cf9-55a47a3b209f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:55.025570 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-692616c7-07e3-4bfd-8cf9-55a47a3b209f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:55.034069 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-692616c7-07e3-4bfd-8cf9-55a47a3b209f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1678 microversion: 2.1 time: 0.178891 Oct 30 01:02:55.035688 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 404/807] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:02:54 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1678 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:02:56.063322 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-a248f064-2951-4978-ae68-b767fe78a866 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:02:56.064813 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-a248f064-2951-4978-ae68-b767fe78a866 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:02:56.073344 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a248f064-2951-4978-ae68-b767fe78a866 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:56.074481 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a248f064-2951-4978-ae68-b767fe78a866 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:56.075070 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a248f064-2951-4978-ae68-b767fe78a866 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:56.076010 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a248f064-2951-4978-ae68-b767fe78a866 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:56.076792 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a248f064-2951-4978-ae68-b767fe78a866 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:56.077402 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a248f064-2951-4978-ae68-b767fe78a866 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:56.232957 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-a248f064-2951-4978-ae68-b767fe78a866 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:56.241577 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-a248f064-2951-4978-ae68-b767fe78a866 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1678 microversion: 2.1 time: 0.182660 Oct 30 01:02:56.242168 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 404/808] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:02:56 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1678 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:02:57.265236 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-1d787105-ad46-472e-b658-e1164d763c02 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:02:57.265927 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-1d787105-ad46-472e-b658-e1164d763c02 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:02:57.273156 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1d787105-ad46-472e-b658-e1164d763c02 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:57.273636 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1d787105-ad46-472e-b658-e1164d763c02 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:57.274197 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1d787105-ad46-472e-b658-e1164d763c02 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:57.276580 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1d787105-ad46-472e-b658-e1164d763c02 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:57.276580 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1d787105-ad46-472e-b658-e1164d763c02 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:57.276580 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1d787105-ad46-472e-b658-e1164d763c02 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:57.431993 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-1d787105-ad46-472e-b658-e1164d763c02 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:57.450725 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-1d787105-ad46-472e-b658-e1164d763c02 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1678 microversion: 2.1 time: 0.189869 Oct 30 01:02:57.452009 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 405/809] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:02:57 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1678 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:02:58.476058 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-b0f405b1-9fe9-424a-bb04-0fa812462bc5 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:02:58.477400 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-b0f405b1-9fe9-424a-bb04-0fa812462bc5 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:02:58.484623 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b0f405b1-9fe9-424a-bb04-0fa812462bc5 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:58.485233 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b0f405b1-9fe9-424a-bb04-0fa812462bc5 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:58.485962 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b0f405b1-9fe9-424a-bb04-0fa812462bc5 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:58.486875 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b0f405b1-9fe9-424a-bb04-0fa812462bc5 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:58.487575 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b0f405b1-9fe9-424a-bb04-0fa812462bc5 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:58.488174 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b0f405b1-9fe9-424a-bb04-0fa812462bc5 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:58.635539 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-b0f405b1-9fe9-424a-bb04-0fa812462bc5 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:58.644645 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-b0f405b1-9fe9-424a-bb04-0fa812462bc5 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1678 microversion: 2.1 time: 0.173828 Oct 30 01:02:58.645369 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 405/810] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:02:58 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1678 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:02:59.668796 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-d7af091e-5d00-46ce-a85d-c3dbb240db05 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:02:59.669369 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-d7af091e-5d00-46ce-a85d-c3dbb240db05 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:02:59.676043 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d7af091e-5d00-46ce-a85d-c3dbb240db05 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:59.676648 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d7af091e-5d00-46ce-a85d-c3dbb240db05 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:59.677165 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d7af091e-5d00-46ce-a85d-c3dbb240db05 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:59.677973 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d7af091e-5d00-46ce-a85d-c3dbb240db05 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:02:59.678387 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d7af091e-5d00-46ce-a85d-c3dbb240db05 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:02:59.678806 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d7af091e-5d00-46ce-a85d-c3dbb240db05 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:02:59.825128 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-d7af091e-5d00-46ce-a85d-c3dbb240db05 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:02:59.833914 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-d7af091e-5d00-46ce-a85d-c3dbb240db05 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1678 microversion: 2.1 time: 0.169123 Oct 30 01:02:59.834523 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 406/811] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:02:59 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1678 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:00.481330 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-f20ec4ec-4576-4810-bc66-8730e8245f2e tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Action: 'create', calling method: .version_select at 0x7f0eaf9ac040>, body: {"keypair": {"name": "tempest-TestMinimumBasicScenario-1028328605", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOCCaIhRWnXAH0ngOg+ZaTyBcpyMaU4Ug6TKfrZtypQwNwInqeBGcw2tp09Ea3+IP1tloFdaIaaiAf1hmItKBaUwotbO5912by3dtyB0Y11MGG4fkh3vDeUBX3AbCyVV2g=="}} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:03:00.498046 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.quota [None req-f20ec4ec-4576-4810-bc66-8730e8245f2e tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Getting quotas for project 1dcbf2e9fb2e4a4698633fcbb07c30cd. Resources: {'key_pairs'} {{(pid=118962) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 30 01:03:00.502104 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.quota [None req-f20ec4ec-4576-4810-bc66-8730e8245f2e tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Getting quotas for user 05eed70effab4e38ae469ab44d1114db and project 1dcbf2e9fb2e4a4698633fcbb07c30cd. Resources: {'key_pairs'} {{(pid=118962) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 30 01:03:00.530492 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-f20ec4ec-4576-4810-bc66-8730e8245f2e tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 403 microversion: 2.1 time: 0.053152 Oct 30 01:03:00.531088 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 406/812] 10.222.0.16 () {60 vars in 1159 bytes} [Sun Oct 30 01:03:00 2022] POST /compute/v2.1/os-keypairs => generated 403 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:03:00.588967 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-1aba1641-67e3-475c-9c01-5461c89dffff tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestMinimumBasicScenario-server-800687934", "imageRef": "39b66fc6-8642-413f-a3b6-735ef1f082ab", "flavorRef": "452", "key_name": "tempest-TestMinimumBasicScenario-1028328605", "networks": [{"uuid": "8d21c8e3-1631-4a7d-967b-2d154077ce1f"}]}} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:03:00.642963 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.network.neutron [None req-1aba1641-67e3-475c-9c01-5461c89dffff tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] validate_networks() for [('8d21c8e3-1631-4a7d-967b-2d154077ce1f', None, None, None, None, None)] {{(pid=118961) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 30 01:03:00.857727 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-c8e3fe2c-b392-4e18-b46c-a2f8eef3f783 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:00.858355 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-c8e3fe2c-b392-4e18-b46c-a2f8eef3f783 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:00.865043 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c8e3fe2c-b392-4e18-b46c-a2f8eef3f783 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:00.865452 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c8e3fe2c-b392-4e18-b46c-a2f8eef3f783 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:00.865879 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c8e3fe2c-b392-4e18-b46c-a2f8eef3f783 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:00.866567 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c8e3fe2c-b392-4e18-b46c-a2f8eef3f783 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:00.867011 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c8e3fe2c-b392-4e18-b46c-a2f8eef3f783 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:00.867425 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c8e3fe2c-b392-4e18-b46c-a2f8eef3f783 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:01.068366 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-c8e3fe2c-b392-4e18-b46c-a2f8eef3f783 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:01.077054 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-c8e3fe2c-b392-4e18-b46c-a2f8eef3f783 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1678 microversion: 2.1 time: 0.223302 Oct 30 01:03:01.077606 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 407/813] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:00 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1678 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:01.093744 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-1aba1641-67e3-475c-9c01-5461c89dffff tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Flavor limits 0:0:0 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 30 01:03:01.094246 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-1aba1641-67e3-475c-9c01-5461c89dffff tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Image limits 0:0:0 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 30 01:03:01.094628 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-1aba1641-67e3-475c-9c01-5461c89dffff tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Flavor pref 0:0:0 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 30 01:03:01.095056 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-1aba1641-67e3-475c-9c01-5461c89dffff tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Image pref 0:0:0 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 30 01:03:01.095470 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-1aba1641-67e3-475c-9c01-5461c89dffff tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 30 01:03:01.104971 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1aba1641-67e3-475c-9c01-5461c89dffff tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:01.105491 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1aba1641-67e3-475c-9c01-5461c89dffff tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:01.105931 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1aba1641-67e3-475c-9c01-5461c89dffff tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:01.106664 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1aba1641-67e3-475c-9c01-5461c89dffff tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:01.107119 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1aba1641-67e3-475c-9c01-5461c89dffff tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:01.107584 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1aba1641-67e3-475c-9c01-5461c89dffff tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:01.125387 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.quota [None req-1aba1641-67e3-475c-9c01-5461c89dffff tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Getting quotas for project 1dcbf2e9fb2e4a4698633fcbb07c30cd. Resources: {'ram', 'instances', 'cores'} {{(pid=118961) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 30 01:03:01.129703 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.quota [None req-1aba1641-67e3-475c-9c01-5461c89dffff tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Getting quotas for user 05eed70effab4e38ae469ab44d1114db and project 1dcbf2e9fb2e4a4698633fcbb07c30cd. Resources: {'ram', 'instances', 'cores'} {{(pid=118961) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 30 01:03:01.141920 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-1aba1641-67e3-475c-9c01-5461c89dffff tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Going to run 1 instances... {{(pid=118961) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 30 01:03:01.144205 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-1aba1641-67e3-475c-9c01-5461c89dffff tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='39b66fc6-8642-413f-a3b6-735ef1f082ab',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118961) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 30 01:03:01.197838 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-1aba1641-67e3-475c-9c01-5461c89dffff tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.612686 Oct 30 01:03:01.198513 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 407/814] 10.222.0.16 () {60 vars in 1147 bytes} [Sun Oct 30 01:03:00 2022] POST /compute/v2.1/servers => generated 380 bytes in 614 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 30 01:03:01.210373 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-108463d4-a341-4114-a41e-b1c8d7db8c26 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:01.210989 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-108463d4-a341-4114-a41e-b1c8d7db8c26 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:01.270784 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-108463d4-a341-4114-a41e-b1c8d7db8c26 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '05eed70effab4e38ae469ab44d1114db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dcbf2e9fb2e4a4698633fcbb07c30cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:01.278604 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-108463d4-a341-4114-a41e-b1c8d7db8c26 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7" status: 200 len: 1258 microversion: 2.1 time: 0.072476 Oct 30 01:03:01.279204 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 408/815] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:01 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7 => generated 1258 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:02.101736 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-5333588f-abc0-4ee1-8f42-3713fa5a0f82 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:02.102345 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-5333588f-abc0-4ee1-8f42-3713fa5a0f82 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:02.109160 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5333588f-abc0-4ee1-8f42-3713fa5a0f82 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:02.109639 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5333588f-abc0-4ee1-8f42-3713fa5a0f82 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:02.110069 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5333588f-abc0-4ee1-8f42-3713fa5a0f82 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:02.110747 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5333588f-abc0-4ee1-8f42-3713fa5a0f82 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:02.111192 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5333588f-abc0-4ee1-8f42-3713fa5a0f82 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:02.111601 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5333588f-abc0-4ee1-8f42-3713fa5a0f82 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:02.253765 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-5333588f-abc0-4ee1-8f42-3713fa5a0f82 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:02.261818 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-5333588f-abc0-4ee1-8f42-3713fa5a0f82 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1678 microversion: 2.1 time: 0.164561 Oct 30 01:03:02.262570 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 408/816] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:02 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1678 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:02.301918 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-224273ba-bb07-4dbc-9371-38a3b7801b3d tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:02.302567 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-224273ba-bb07-4dbc-9371-38a3b7801b3d tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:02.308474 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-224273ba-bb07-4dbc-9371-38a3b7801b3d tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:02.309036 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-224273ba-bb07-4dbc-9371-38a3b7801b3d tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:02.309454 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-224273ba-bb07-4dbc-9371-38a3b7801b3d tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:02.310139 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-224273ba-bb07-4dbc-9371-38a3b7801b3d tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:02.310580 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-224273ba-bb07-4dbc-9371-38a3b7801b3d tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:02.310987 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-224273ba-bb07-4dbc-9371-38a3b7801b3d tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:02.371977 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-224273ba-bb07-4dbc-9371-38a3b7801b3d tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '05eed70effab4e38ae469ab44d1114db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dcbf2e9fb2e4a4698633fcbb07c30cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:02.379950 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-224273ba-bb07-4dbc-9371-38a3b7801b3d tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7" status: 200 len: 1310 microversion: 2.1 time: 0.082047 Oct 30 01:03:02.380473 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 409/817] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:02 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7 => generated 1310 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:03.286600 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-9b2f838b-c2eb-4b81-ac88-f4d0c35d41d7 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:03.288031 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-9b2f838b-c2eb-4b81-ac88-f4d0c35d41d7 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:03.295627 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9b2f838b-c2eb-4b81-ac88-f4d0c35d41d7 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:03.296390 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9b2f838b-c2eb-4b81-ac88-f4d0c35d41d7 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:03.297067 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9b2f838b-c2eb-4b81-ac88-f4d0c35d41d7 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:03.298130 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9b2f838b-c2eb-4b81-ac88-f4d0c35d41d7 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:03.298741 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9b2f838b-c2eb-4b81-ac88-f4d0c35d41d7 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:03.299337 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9b2f838b-c2eb-4b81-ac88-f4d0c35d41d7 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:03.404966 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-986e9330-ae2a-4aff-ba44-2f8ae033e5eb tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:03.406374 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-986e9330-ae2a-4aff-ba44-2f8ae033e5eb tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:03.414253 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-986e9330-ae2a-4aff-ba44-2f8ae033e5eb tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:03.414830 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-986e9330-ae2a-4aff-ba44-2f8ae033e5eb tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:03.415270 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-986e9330-ae2a-4aff-ba44-2f8ae033e5eb tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:03.416047 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-986e9330-ae2a-4aff-ba44-2f8ae033e5eb tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:03.416505 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-986e9330-ae2a-4aff-ba44-2f8ae033e5eb tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:03.417118 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-986e9330-ae2a-4aff-ba44-2f8ae033e5eb tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:03.551298 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-986e9330-ae2a-4aff-ba44-2f8ae033e5eb tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '05eed70effab4e38ae469ab44d1114db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dcbf2e9fb2e4a4698633fcbb07c30cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:03.563747 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-986e9330-ae2a-4aff-ba44-2f8ae033e5eb tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7" status: 200 len: 1316 microversion: 2.1 time: 0.164860 Oct 30 01:03:03.564683 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 410/818] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:03 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7 => generated 1316 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 01:03:03.569226 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-9b2f838b-c2eb-4b81-ac88-f4d0c35d41d7 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:03.579375 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-9b2f838b-c2eb-4b81-ac88-f4d0c35d41d7 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1678 microversion: 2.1 time: 0.296703 Oct 30 01:03:03.584477 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 409/819] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:03 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1678 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:04.587869 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-0934ffb7-89bb-494e-a682-9fd5dc10d6ed tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:04.588517 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-0934ffb7-89bb-494e-a682-9fd5dc10d6ed tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:04.596351 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0934ffb7-89bb-494e-a682-9fd5dc10d6ed tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:04.596761 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0934ffb7-89bb-494e-a682-9fd5dc10d6ed tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:04.597256 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0934ffb7-89bb-494e-a682-9fd5dc10d6ed tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:04.598031 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0934ffb7-89bb-494e-a682-9fd5dc10d6ed tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:04.598591 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0934ffb7-89bb-494e-a682-9fd5dc10d6ed tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:04.599023 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0934ffb7-89bb-494e-a682-9fd5dc10d6ed tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:04.608499 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-372a0296-d6aa-4681-854c-c3de217cfb99 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:04.609623 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-372a0296-d6aa-4681-854c-c3de217cfb99 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:04.616751 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-372a0296-d6aa-4681-854c-c3de217cfb99 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:04.617244 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-372a0296-d6aa-4681-854c-c3de217cfb99 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:04.617686 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-372a0296-d6aa-4681-854c-c3de217cfb99 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:04.618472 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-372a0296-d6aa-4681-854c-c3de217cfb99 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:04.618841 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-372a0296-d6aa-4681-854c-c3de217cfb99 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:04.619274 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-372a0296-d6aa-4681-854c-c3de217cfb99 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:04.814719 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-0934ffb7-89bb-494e-a682-9fd5dc10d6ed tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '05eed70effab4e38ae469ab44d1114db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dcbf2e9fb2e4a4698633fcbb07c30cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:04.818415 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-372a0296-d6aa-4681-854c-c3de217cfb99 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:04.824729 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-0934ffb7-89bb-494e-a682-9fd5dc10d6ed tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7" status: 200 len: 1358 microversion: 2.1 time: 0.240806 Oct 30 01:03:04.827306 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-372a0296-d6aa-4681-854c-c3de217cfb99 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1678 microversion: 2.1 time: 0.223153 Oct 30 01:03:04.828451 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 410/820] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:04 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1678 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:04.837937 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 411/821] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:04 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7 => generated 1358 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:05.851333 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-7bcb854d-02ce-46d3-ba11-403eb9b1b1b5 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:05.854850 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-275377d1-6927-452f-8b27-8f4877444590 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:05.855446 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-275377d1-6927-452f-8b27-8f4877444590 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:05.855983 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-7bcb854d-02ce-46d3-ba11-403eb9b1b1b5 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:05.866062 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7bcb854d-02ce-46d3-ba11-403eb9b1b1b5 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:05.866296 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-275377d1-6927-452f-8b27-8f4877444590 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:05.866527 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7bcb854d-02ce-46d3-ba11-403eb9b1b1b5 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:05.866762 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-275377d1-6927-452f-8b27-8f4877444590 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:05.866987 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7bcb854d-02ce-46d3-ba11-403eb9b1b1b5 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:05.867236 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-275377d1-6927-452f-8b27-8f4877444590 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:05.867686 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7bcb854d-02ce-46d3-ba11-403eb9b1b1b5 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:05.867976 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-275377d1-6927-452f-8b27-8f4877444590 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:05.868155 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7bcb854d-02ce-46d3-ba11-403eb9b1b1b5 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:05.868428 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-275377d1-6927-452f-8b27-8f4877444590 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:05.868570 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7bcb854d-02ce-46d3-ba11-403eb9b1b1b5 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:05.868837 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-275377d1-6927-452f-8b27-8f4877444590 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:06.029005 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-275377d1-6927-452f-8b27-8f4877444590 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '05eed70effab4e38ae469ab44d1114db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dcbf2e9fb2e4a4698633fcbb07c30cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:06.038996 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-275377d1-6927-452f-8b27-8f4877444590 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7" status: 200 len: 1358 microversion: 2.1 time: 0.194033 Oct 30 01:03:06.040364 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 411/822] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:05 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7 => generated 1358 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:06.162342 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-7bcb854d-02ce-46d3-ba11-403eb9b1b1b5 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:06.170482 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-7bcb854d-02ce-46d3-ba11-403eb9b1b1b5 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1678 microversion: 2.1 time: 0.322907 Oct 30 01:03:06.171058 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 412/823] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:05 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1678 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:07.064602 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-92ec008e-8725-4af3-8632-5d7ae585d7c3 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:07.065885 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-92ec008e-8725-4af3-8632-5d7ae585d7c3 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:07.072546 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-92ec008e-8725-4af3-8632-5d7ae585d7c3 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:07.073035 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-92ec008e-8725-4af3-8632-5d7ae585d7c3 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:07.073464 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-92ec008e-8725-4af3-8632-5d7ae585d7c3 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:07.074172 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-92ec008e-8725-4af3-8632-5d7ae585d7c3 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:07.074625 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-92ec008e-8725-4af3-8632-5d7ae585d7c3 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:07.075047 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-92ec008e-8725-4af3-8632-5d7ae585d7c3 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:07.194619 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-39fffd1a-68d5-4ee5-bce8-aa1c27615a62 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:07.195377 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-39fffd1a-68d5-4ee5-bce8-aa1c27615a62 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:07.202049 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-39fffd1a-68d5-4ee5-bce8-aa1c27615a62 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:07.202618 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-39fffd1a-68d5-4ee5-bce8-aa1c27615a62 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:07.203105 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-39fffd1a-68d5-4ee5-bce8-aa1c27615a62 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:07.203932 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-39fffd1a-68d5-4ee5-bce8-aa1c27615a62 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:07.204453 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-39fffd1a-68d5-4ee5-bce8-aa1c27615a62 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:07.204963 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-39fffd1a-68d5-4ee5-bce8-aa1c27615a62 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:07.219572 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-92ec008e-8725-4af3-8632-5d7ae585d7c3 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '05eed70effab4e38ae469ab44d1114db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dcbf2e9fb2e4a4698633fcbb07c30cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:07.228010 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-92ec008e-8725-4af3-8632-5d7ae585d7c3 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7" status: 200 len: 1358 microversion: 2.1 time: 0.168261 Oct 30 01:03:07.243493 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 412/824] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:07 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7 => generated 1358 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:07.339871 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-39fffd1a-68d5-4ee5-bce8-aa1c27615a62 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:07.351335 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-39fffd1a-68d5-4ee5-bce8-aa1c27615a62 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1678 microversion: 2.1 time: 0.160680 Oct 30 01:03:07.351953 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 413/825] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:07 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1678 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 01:03:08.255001 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-ecfc96fd-79e3-43dc-bf03-5650ba394639 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:08.256154 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-ecfc96fd-79e3-43dc-bf03-5650ba394639 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:08.267129 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ecfc96fd-79e3-43dc-bf03-5650ba394639 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:08.268047 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ecfc96fd-79e3-43dc-bf03-5650ba394639 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:08.268641 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ecfc96fd-79e3-43dc-bf03-5650ba394639 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:08.269460 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ecfc96fd-79e3-43dc-bf03-5650ba394639 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:08.270005 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ecfc96fd-79e3-43dc-bf03-5650ba394639 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:08.270433 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ecfc96fd-79e3-43dc-bf03-5650ba394639 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:08.374676 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-9ab09ecc-0c8f-494b-87d7-902c85454be0 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:08.375290 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-9ab09ecc-0c8f-494b-87d7-902c85454be0 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:08.387995 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9ab09ecc-0c8f-494b-87d7-902c85454be0 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:08.388502 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9ab09ecc-0c8f-494b-87d7-902c85454be0 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:08.389707 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9ab09ecc-0c8f-494b-87d7-902c85454be0 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:08.390447 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9ab09ecc-0c8f-494b-87d7-902c85454be0 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:08.390893 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9ab09ecc-0c8f-494b-87d7-902c85454be0 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:08.391315 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9ab09ecc-0c8f-494b-87d7-902c85454be0 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:08.430326 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-ecfc96fd-79e3-43dc-bf03-5650ba394639 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '05eed70effab4e38ae469ab44d1114db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dcbf2e9fb2e4a4698633fcbb07c30cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:08.438248 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-ecfc96fd-79e3-43dc-bf03-5650ba394639 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7" status: 200 len: 1358 microversion: 2.1 time: 0.189787 Oct 30 01:03:08.439029 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 413/826] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:08 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7 => generated 1358 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 01:03:08.540045 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-9ab09ecc-0c8f-494b-87d7-902c85454be0 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:08.547941 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-9ab09ecc-0c8f-494b-87d7-902c85454be0 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1678 microversion: 2.1 time: 0.177355 Oct 30 01:03:08.548505 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 414/827] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:08 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1678 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:09.462083 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-d051a1da-849b-49d9-9ac6-64d7b1e52d9f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:09.462725 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-d051a1da-849b-49d9-9ac6-64d7b1e52d9f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:09.469616 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d051a1da-849b-49d9-9ac6-64d7b1e52d9f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:09.470105 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d051a1da-849b-49d9-9ac6-64d7b1e52d9f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:09.470537 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d051a1da-849b-49d9-9ac6-64d7b1e52d9f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:09.471253 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d051a1da-849b-49d9-9ac6-64d7b1e52d9f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:09.471786 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d051a1da-849b-49d9-9ac6-64d7b1e52d9f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:09.472219 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d051a1da-849b-49d9-9ac6-64d7b1e52d9f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:09.576182 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-99bf12ac-2016-4fe2-9a5b-ffc5a45ee12b tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:09.576959 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-99bf12ac-2016-4fe2-9a5b-ffc5a45ee12b tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:09.586400 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-99bf12ac-2016-4fe2-9a5b-ffc5a45ee12b tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:09.593007 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-99bf12ac-2016-4fe2-9a5b-ffc5a45ee12b tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:09.593566 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-99bf12ac-2016-4fe2-9a5b-ffc5a45ee12b tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:09.594421 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-99bf12ac-2016-4fe2-9a5b-ffc5a45ee12b tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:09.594998 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-99bf12ac-2016-4fe2-9a5b-ffc5a45ee12b tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:09.597374 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-99bf12ac-2016-4fe2-9a5b-ffc5a45ee12b tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:09.615346 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-d051a1da-849b-49d9-9ac6-64d7b1e52d9f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '05eed70effab4e38ae469ab44d1114db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dcbf2e9fb2e4a4698633fcbb07c30cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:09.624011 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-d051a1da-849b-49d9-9ac6-64d7b1e52d9f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7" status: 200 len: 1358 microversion: 2.1 time: 0.165941 Oct 30 01:03:09.624602 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 414/828] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:09 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7 => generated 1358 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:09.741813 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-99bf12ac-2016-4fe2-9a5b-ffc5a45ee12b tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:09.749761 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-99bf12ac-2016-4fe2-9a5b-ffc5a45ee12b tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1678 microversion: 2.1 time: 0.180993 Oct 30 01:03:09.750320 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 415/829] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:09 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1678 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:10.648752 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-1da69484-e7b1-4169-8d49-1c0013989863 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:10.649300 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-1da69484-e7b1-4169-8d49-1c0013989863 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:10.657436 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1da69484-e7b1-4169-8d49-1c0013989863 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:10.657989 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1da69484-e7b1-4169-8d49-1c0013989863 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:10.658427 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1da69484-e7b1-4169-8d49-1c0013989863 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:10.659119 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1da69484-e7b1-4169-8d49-1c0013989863 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:10.659740 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1da69484-e7b1-4169-8d49-1c0013989863 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:10.660186 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1da69484-e7b1-4169-8d49-1c0013989863 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:10.775002 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-bc1cd554-1e86-4e6a-b353-77bf622a11e7 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:10.776281 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-bc1cd554-1e86-4e6a-b353-77bf622a11e7 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:10.784742 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bc1cd554-1e86-4e6a-b353-77bf622a11e7 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:10.785731 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bc1cd554-1e86-4e6a-b353-77bf622a11e7 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:10.786443 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bc1cd554-1e86-4e6a-b353-77bf622a11e7 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:10.787511 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bc1cd554-1e86-4e6a-b353-77bf622a11e7 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:10.788182 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bc1cd554-1e86-4e6a-b353-77bf622a11e7 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:10.788934 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bc1cd554-1e86-4e6a-b353-77bf622a11e7 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:10.814750 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-1da69484-e7b1-4169-8d49-1c0013989863 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '05eed70effab4e38ae469ab44d1114db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dcbf2e9fb2e4a4698633fcbb07c30cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:10.823293 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-1da69484-e7b1-4169-8d49-1c0013989863 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7" status: 200 len: 1358 microversion: 2.1 time: 0.178454 Oct 30 01:03:10.840216 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 415/830] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:10 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7 => generated 1358 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:10.940554 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-bc1cd554-1e86-4e6a-b353-77bf622a11e7 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:10.949282 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-bc1cd554-1e86-4e6a-b353-77bf622a11e7 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1678 microversion: 2.1 time: 0.179838 Oct 30 01:03:10.949870 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 416/831] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:10 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1678 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:11.847557 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-085cb7e7-7cfa-4d75-8a4c-19108449064f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:11.848224 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-085cb7e7-7cfa-4d75-8a4c-19108449064f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:11.850585 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell0:SELECT=2 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:03:11.851256 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell0:SELECT=6 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:03:11.857361 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-085cb7e7-7cfa-4d75-8a4c-19108449064f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:11.857854 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-085cb7e7-7cfa-4d75-8a4c-19108449064f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:11.858262 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-085cb7e7-7cfa-4d75-8a4c-19108449064f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:11.858979 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-085cb7e7-7cfa-4d75-8a4c-19108449064f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:11.859424 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-085cb7e7-7cfa-4d75-8a4c-19108449064f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:11.859881 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-085cb7e7-7cfa-4d75-8a4c-19108449064f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:11.973846 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-c36a8c6f-45b6-44ec-b824-490fdf47fefe tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:11.974699 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-c36a8c6f-45b6-44ec-b824-490fdf47fefe tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:11.978372 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell0:SELECT=6 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:03:11.983815 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c36a8c6f-45b6-44ec-b824-490fdf47fefe tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:11.984376 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c36a8c6f-45b6-44ec-b824-490fdf47fefe tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:11.984981 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c36a8c6f-45b6-44ec-b824-490fdf47fefe tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:11.985688 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c36a8c6f-45b6-44ec-b824-490fdf47fefe tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:11.986130 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c36a8c6f-45b6-44ec-b824-490fdf47fefe tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:11.986547 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c36a8c6f-45b6-44ec-b824-490fdf47fefe tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:12.008116 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-085cb7e7-7cfa-4d75-8a4c-19108449064f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '05eed70effab4e38ae469ab44d1114db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dcbf2e9fb2e4a4698633fcbb07c30cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:12.026211 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-085cb7e7-7cfa-4d75-8a4c-19108449064f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7" status: 200 len: 1358 microversion: 2.1 time: 0.183028 Oct 30 01:03:12.036391 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 416/832] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:11 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7 => generated 1358 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:12.131110 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-c36a8c6f-45b6-44ec-b824-490fdf47fefe tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:12.139093 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-c36a8c6f-45b6-44ec-b824-490fdf47fefe tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1678 microversion: 2.1 time: 0.169488 Oct 30 01:03:12.139664 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 417/833] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:11 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1678 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 01:03:13.050905 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-74c3d93f-cbfc-4cbf-933e-12997e5fe322 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:13.051581 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-74c3d93f-cbfc-4cbf-933e-12997e5fe322 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:13.058035 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-74c3d93f-cbfc-4cbf-933e-12997e5fe322 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:13.058522 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-74c3d93f-cbfc-4cbf-933e-12997e5fe322 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:13.058949 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-74c3d93f-cbfc-4cbf-933e-12997e5fe322 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:13.059652 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-74c3d93f-cbfc-4cbf-933e-12997e5fe322 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:13.060115 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-74c3d93f-cbfc-4cbf-933e-12997e5fe322 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:13.060541 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-74c3d93f-cbfc-4cbf-933e-12997e5fe322 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:13.162524 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-f91d075b-b28e-4424-b486-704ecfabb59c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:13.163299 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-f91d075b-b28e-4424-b486-704ecfabb59c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:13.169956 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f91d075b-b28e-4424-b486-704ecfabb59c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:13.170440 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f91d075b-b28e-4424-b486-704ecfabb59c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:13.170863 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f91d075b-b28e-4424-b486-704ecfabb59c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:13.171600 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f91d075b-b28e-4424-b486-704ecfabb59c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:13.172075 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f91d075b-b28e-4424-b486-704ecfabb59c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:13.172488 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f91d075b-b28e-4424-b486-704ecfabb59c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:13.204992 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-74c3d93f-cbfc-4cbf-933e-12997e5fe322 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '05eed70effab4e38ae469ab44d1114db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dcbf2e9fb2e4a4698633fcbb07c30cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:13.213693 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-74c3d93f-cbfc-4cbf-933e-12997e5fe322 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7" status: 200 len: 1358 microversion: 2.1 time: 0.166799 Oct 30 01:03:13.214379 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 417/834] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:13 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7 => generated 1358 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:13.325755 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-f91d075b-b28e-4424-b486-704ecfabb59c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:13.334473 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-f91d075b-b28e-4424-b486-704ecfabb59c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1678 microversion: 2.1 time: 0.175915 Oct 30 01:03:13.335073 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 418/835] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:13 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1678 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 01:03:14.241746 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-b813086c-3e0f-4ddf-af3c-311d46105079 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:14.242617 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-b813086c-3e0f-4ddf-af3c-311d46105079 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:14.253611 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b813086c-3e0f-4ddf-af3c-311d46105079 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:14.254443 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b813086c-3e0f-4ddf-af3c-311d46105079 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:14.255155 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b813086c-3e0f-4ddf-af3c-311d46105079 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:14.256664 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b813086c-3e0f-4ddf-af3c-311d46105079 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:14.257135 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b813086c-3e0f-4ddf-af3c-311d46105079 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:14.257552 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b813086c-3e0f-4ddf-af3c-311d46105079 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:14.357818 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-6cb684ae-f3cc-419d-a638-020f1563f3e9 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:14.358586 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-6cb684ae-f3cc-419d-a638-020f1563f3e9 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:14.365603 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6cb684ae-f3cc-419d-a638-020f1563f3e9 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:14.366137 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6cb684ae-f3cc-419d-a638-020f1563f3e9 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:14.366768 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6cb684ae-f3cc-419d-a638-020f1563f3e9 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:14.367634 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6cb684ae-f3cc-419d-a638-020f1563f3e9 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:14.368253 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6cb684ae-f3cc-419d-a638-020f1563f3e9 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:14.368882 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6cb684ae-f3cc-419d-a638-020f1563f3e9 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:14.396350 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-b813086c-3e0f-4ddf-af3c-311d46105079 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '05eed70effab4e38ae469ab44d1114db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dcbf2e9fb2e4a4698633fcbb07c30cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:14.404288 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-b813086c-3e0f-4ddf-af3c-311d46105079 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7" status: 200 len: 1358 microversion: 2.1 time: 0.169443 Oct 30 01:03:14.404867 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 418/836] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:14 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7 => generated 1358 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:14.496934 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-6cb684ae-f3cc-419d-a638-020f1563f3e9 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:14.505022 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-6cb684ae-f3cc-419d-a638-020f1563f3e9 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1678 microversion: 2.1 time: 0.151097 Oct 30 01:03:14.505579 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 419/837] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:14 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1678 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:15.429953 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-66da8085-4270-4d69-bfe6-7c6d103cf62b tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:15.430590 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-66da8085-4270-4d69-bfe6-7c6d103cf62b tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:15.437744 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-66da8085-4270-4d69-bfe6-7c6d103cf62b tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:15.438224 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-66da8085-4270-4d69-bfe6-7c6d103cf62b tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:15.438658 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-66da8085-4270-4d69-bfe6-7c6d103cf62b tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:15.439352 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-66da8085-4270-4d69-bfe6-7c6d103cf62b tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:15.439837 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-66da8085-4270-4d69-bfe6-7c6d103cf62b tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:15.440259 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-66da8085-4270-4d69-bfe6-7c6d103cf62b tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:15.458667 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:03:15.528924 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-2a27bc25-2afb-4fcc-a19f-da86771f0db5 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:15.529525 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-2a27bc25-2afb-4fcc-a19f-da86771f0db5 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:15.536508 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2a27bc25-2afb-4fcc-a19f-da86771f0db5 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:15.536987 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2a27bc25-2afb-4fcc-a19f-da86771f0db5 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:15.537423 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2a27bc25-2afb-4fcc-a19f-da86771f0db5 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:15.538120 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2a27bc25-2afb-4fcc-a19f-da86771f0db5 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:15.538635 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2a27bc25-2afb-4fcc-a19f-da86771f0db5 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:15.539114 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2a27bc25-2afb-4fcc-a19f-da86771f0db5 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:15.558880 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:03:15.576531 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-66da8085-4270-4d69-bfe6-7c6d103cf62b tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '05eed70effab4e38ae469ab44d1114db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dcbf2e9fb2e4a4698633fcbb07c30cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:15.585041 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-66da8085-4270-4d69-bfe6-7c6d103cf62b tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7" status: 200 len: 1358 microversion: 2.1 time: 0.160169 Oct 30 01:03:15.585749 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 419/838] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:15 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7 => generated 1358 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:15.679840 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-2a27bc25-2afb-4fcc-a19f-da86771f0db5 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:15.689030 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-2a27bc25-2afb-4fcc-a19f-da86771f0db5 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1678 microversion: 2.1 time: 0.164128 Oct 30 01:03:15.689610 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 420/839] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:15 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1678 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:16.608786 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-7d9d3628-aa49-4fe9-9155-84467fea6244 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:16.609350 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-7d9d3628-aa49-4fe9-9155-84467fea6244 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:16.617129 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7d9d3628-aa49-4fe9-9155-84467fea6244 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:16.617129 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7d9d3628-aa49-4fe9-9155-84467fea6244 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:16.617552 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7d9d3628-aa49-4fe9-9155-84467fea6244 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:16.618918 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7d9d3628-aa49-4fe9-9155-84467fea6244 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:16.618918 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7d9d3628-aa49-4fe9-9155-84467fea6244 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:16.619112 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7d9d3628-aa49-4fe9-9155-84467fea6244 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:16.714338 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-7eeb07ec-9f11-4f71-8bc3-a5e2faa63801 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:16.714923 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-7eeb07ec-9f11-4f71-8bc3-a5e2faa63801 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:16.721442 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7eeb07ec-9f11-4f71-8bc3-a5e2faa63801 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:16.721924 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7eeb07ec-9f11-4f71-8bc3-a5e2faa63801 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:16.722564 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7eeb07ec-9f11-4f71-8bc3-a5e2faa63801 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:16.723026 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7eeb07ec-9f11-4f71-8bc3-a5e2faa63801 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:16.723546 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7eeb07ec-9f11-4f71-8bc3-a5e2faa63801 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:16.723895 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7eeb07ec-9f11-4f71-8bc3-a5e2faa63801 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:16.805933 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-7d9d3628-aa49-4fe9-9155-84467fea6244 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '05eed70effab4e38ae469ab44d1114db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dcbf2e9fb2e4a4698633fcbb07c30cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:16.814001 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-7d9d3628-aa49-4fe9-9155-84467fea6244 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7" status: 200 len: 1358 microversion: 2.1 time: 0.209398 Oct 30 01:03:16.815899 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 420/840] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:16 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7 => generated 1358 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:16.867234 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-7eeb07ec-9f11-4f71-8bc3-a5e2faa63801 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:16.875487 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-7eeb07ec-9f11-4f71-8bc3-a5e2faa63801 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1678 microversion: 2.1 time: 0.165145 Oct 30 01:03:16.876209 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 421/841] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:16 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1678 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:17.838724 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-025792e3-3943-4179-a093-473bb42d147d tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:17.839340 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-025792e3-3943-4179-a093-473bb42d147d tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:17.846271 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-025792e3-3943-4179-a093-473bb42d147d tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:17.846762 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-025792e3-3943-4179-a093-473bb42d147d tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:17.847202 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-025792e3-3943-4179-a093-473bb42d147d tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:17.847947 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-025792e3-3943-4179-a093-473bb42d147d tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:17.848403 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-025792e3-3943-4179-a093-473bb42d147d tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:17.848855 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-025792e3-3943-4179-a093-473bb42d147d tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:17.898742 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-ee095993-e148-42cb-9af3-ee178a540d44 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:17.899406 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-ee095993-e148-42cb-9af3-ee178a540d44 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:17.905847 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ee095993-e148-42cb-9af3-ee178a540d44 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:17.906389 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ee095993-e148-42cb-9af3-ee178a540d44 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:17.906903 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ee095993-e148-42cb-9af3-ee178a540d44 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:17.907616 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ee095993-e148-42cb-9af3-ee178a540d44 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:17.908171 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ee095993-e148-42cb-9af3-ee178a540d44 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:17.908620 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ee095993-e148-42cb-9af3-ee178a540d44 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:18.036111 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-025792e3-3943-4179-a093-473bb42d147d tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '05eed70effab4e38ae469ab44d1114db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dcbf2e9fb2e4a4698633fcbb07c30cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:18.045268 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-025792e3-3943-4179-a093-473bb42d147d tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7" status: 200 len: 1358 microversion: 2.1 time: 0.210884 Oct 30 01:03:18.046463 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 421/842] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:17 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7 => generated 1358 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:18.084537 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-ee095993-e148-42cb-9af3-ee178a540d44 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:18.094271 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-ee095993-e148-42cb-9af3-ee178a540d44 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1678 microversion: 2.1 time: 0.199452 Oct 30 01:03:18.094850 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 422/843] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:17 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1678 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:18.563208 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [req-fa41f5bc-91dd-48f0-899b-85be13e0bdce req-1bd15a85-e43b-451a-bd83-1d0b98bb3ddc service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6f5485fd-b825-45e7-a5be-eb96208c09b7", "tag": "ad10e942-b527-4fbd-867f-e161a78a11bc"}]} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:03:18.580725 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-fa41f5bc-91dd-48f0-899b-85be13e0bdce req-1bd15a85-e43b-451a-bd83-1d0b98bb3ddc service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:18.581403 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-fa41f5bc-91dd-48f0-899b-85be13e0bdce req-1bd15a85-e43b-451a-bd83-1d0b98bb3ddc service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:18.582016 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-fa41f5bc-91dd-48f0-899b-85be13e0bdce req-1bd15a85-e43b-451a-bd83-1d0b98bb3ddc service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:18.595248 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.compute.server_external_events [req-fa41f5bc-91dd-48f0-899b-85be13e0bdce req-1bd15a85-e43b-451a-bd83-1d0b98bb3ddc service nova] Creating event network-changed:ad10e942-b527-4fbd-867f-e161a78a11bc for instance 6f5485fd-b825-45e7-a5be-eb96208c09b7 on hv-cinder-maste Oct 30 01:03:18.600960 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [req-fa41f5bc-91dd-48f0-899b-85be13e0bdce req-1bd15a85-e43b-451a-bd83-1d0b98bb3ddc service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.043553 Oct 30 01:03:18.602168 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 422/844] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:03:18 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:03:19.071666 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-e8efcd79-18fd-4785-86ae-393290b80d7f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:19.072318 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-e8efcd79-18fd-4785-86ae-393290b80d7f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:19.079347 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e8efcd79-18fd-4785-86ae-393290b80d7f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:19.079869 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e8efcd79-18fd-4785-86ae-393290b80d7f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:19.080310 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e8efcd79-18fd-4785-86ae-393290b80d7f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:19.081036 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e8efcd79-18fd-4785-86ae-393290b80d7f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:19.081492 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e8efcd79-18fd-4785-86ae-393290b80d7f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:19.081917 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e8efcd79-18fd-4785-86ae-393290b80d7f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:19.119183 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-eafe516d-4b7f-476d-b826-19c7e61aecc7 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:19.120207 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-eafe516d-4b7f-476d-b826-19c7e61aecc7 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:19.126998 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-eafe516d-4b7f-476d-b826-19c7e61aecc7 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:19.127573 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-eafe516d-4b7f-476d-b826-19c7e61aecc7 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:19.128100 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-eafe516d-4b7f-476d-b826-19c7e61aecc7 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:19.128903 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-eafe516d-4b7f-476d-b826-19c7e61aecc7 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:19.129582 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-eafe516d-4b7f-476d-b826-19c7e61aecc7 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:19.130233 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-eafe516d-4b7f-476d-b826-19c7e61aecc7 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:19.267555 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-e8efcd79-18fd-4785-86ae-393290b80d7f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '05eed70effab4e38ae469ab44d1114db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dcbf2e9fb2e4a4698633fcbb07c30cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:19.276759 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-e8efcd79-18fd-4785-86ae-393290b80d7f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7" status: 200 len: 1358 microversion: 2.1 time: 0.209037 Oct 30 01:03:19.277794 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 423/845] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:19 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7 => generated 1358 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:19.307218 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-eafe516d-4b7f-476d-b826-19c7e61aecc7 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:19.315970 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-eafe516d-4b7f-476d-b826-19c7e61aecc7 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1655 microversion: 2.1 time: 0.201108 Oct 30 01:03:19.316543 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 423/846] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:19 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1655 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:19.339053 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-57719154-3dbb-46f7-a107-acecdeabbb12 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:03:19.345313 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-57719154-3dbb-46f7-a107-acecdeabbb12 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:19.345806 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-57719154-3dbb-46f7-a107-acecdeabbb12 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:19.346239 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-57719154-3dbb-46f7-a107-acecdeabbb12 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:19.369178 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-57719154-3dbb-46f7-a107-acecdeabbb12 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Going to try to start instance {{(pid=118962) start /opt/stack/nova/nova/compute/api.py:2785}} Oct 30 01:03:19.415891 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-57719154-3dbb-46f7-a107-acecdeabbb12 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "POST /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14/action" status: 202 len: 0 microversion: 2.1 time: 0.080874 Oct 30 01:03:19.416494 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 424/847] 10.222.0.16 () {60 vars in 1278 bytes} [Sun Oct 30 01:03:19 2022] POST /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14/action => generated 0 bytes in 82 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 30 01:03:19.425021 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-7d4ea2bf-2a03-49b6-92cd-b38d70c79e89 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:19.425637 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-7d4ea2bf-2a03-49b6-92cd-b38d70c79e89 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:19.431636 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7d4ea2bf-2a03-49b6-92cd-b38d70c79e89 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:19.432182 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7d4ea2bf-2a03-49b6-92cd-b38d70c79e89 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:19.432626 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7d4ea2bf-2a03-49b6-92cd-b38d70c79e89 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:19.433283 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7d4ea2bf-2a03-49b6-92cd-b38d70c79e89 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:19.433758 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7d4ea2bf-2a03-49b6-92cd-b38d70c79e89 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:19.434371 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7d4ea2bf-2a03-49b6-92cd-b38d70c79e89 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:19.577482 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-7d4ea2bf-2a03-49b6-92cd-b38d70c79e89 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:19.589876 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-7d4ea2bf-2a03-49b6-92cd-b38d70c79e89 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1664 microversion: 2.1 time: 0.168594 Oct 30 01:03:19.590727 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 424/848] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:19 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1664 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:20.306033 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-5901bfd4-8e59-467b-bfa8-9992956fd4ae tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:20.307216 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-5901bfd4-8e59-467b-bfa8-9992956fd4ae tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:20.317712 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5901bfd4-8e59-467b-bfa8-9992956fd4ae tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:20.318540 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5901bfd4-8e59-467b-bfa8-9992956fd4ae tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:20.319254 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5901bfd4-8e59-467b-bfa8-9992956fd4ae tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:20.320351 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5901bfd4-8e59-467b-bfa8-9992956fd4ae tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:20.320797 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5901bfd4-8e59-467b-bfa8-9992956fd4ae tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:20.321215 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5901bfd4-8e59-467b-bfa8-9992956fd4ae tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:20.456730 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-5901bfd4-8e59-467b-bfa8-9992956fd4ae tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '05eed70effab4e38ae469ab44d1114db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dcbf2e9fb2e4a4698633fcbb07c30cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:20.468414 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-5901bfd4-8e59-467b-bfa8-9992956fd4ae tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7" status: 200 len: 1358 microversion: 2.1 time: 0.169898 Oct 30 01:03:20.469233 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 425/849] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:20 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7 => generated 1358 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:20.622496 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-26dce885-7872-47d9-a342-ba32a2ad196a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:20.623345 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-26dce885-7872-47d9-a342-ba32a2ad196a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:20.634507 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-26dce885-7872-47d9-a342-ba32a2ad196a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:20.635305 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-26dce885-7872-47d9-a342-ba32a2ad196a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:20.636075 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-26dce885-7872-47d9-a342-ba32a2ad196a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:20.637275 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-26dce885-7872-47d9-a342-ba32a2ad196a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:20.638019 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-26dce885-7872-47d9-a342-ba32a2ad196a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:20.638723 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-26dce885-7872-47d9-a342-ba32a2ad196a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:20.775922 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-26dce885-7872-47d9-a342-ba32a2ad196a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:20.783783 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-26dce885-7872-47d9-a342-ba32a2ad196a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1664 microversion: 2.1 time: 0.168510 Oct 30 01:03:20.784384 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 425/850] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:20 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1664 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:21.498907 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-dd750c28-c05b-41cd-b0f7-6604df0300f0 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:21.499781 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-dd750c28-c05b-41cd-b0f7-6604df0300f0 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:21.510996 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dd750c28-c05b-41cd-b0f7-6604df0300f0 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:21.511932 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dd750c28-c05b-41cd-b0f7-6604df0300f0 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:21.512676 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dd750c28-c05b-41cd-b0f7-6604df0300f0 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:21.513890 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dd750c28-c05b-41cd-b0f7-6604df0300f0 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:21.514645 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dd750c28-c05b-41cd-b0f7-6604df0300f0 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:21.515330 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dd750c28-c05b-41cd-b0f7-6604df0300f0 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:21.664228 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-dd750c28-c05b-41cd-b0f7-6604df0300f0 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '05eed70effab4e38ae469ab44d1114db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dcbf2e9fb2e4a4698633fcbb07c30cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:21.672918 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-dd750c28-c05b-41cd-b0f7-6604df0300f0 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7" status: 200 len: 1358 microversion: 2.1 time: 0.180784 Oct 30 01:03:21.673479 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 426/851] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:21 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7 => generated 1358 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:21.810843 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-42bbf9df-37f1-4d0b-81de-61c4fff8255e tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:21.811877 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-42bbf9df-37f1-4d0b-81de-61c4fff8255e tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:21.822407 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-42bbf9df-37f1-4d0b-81de-61c4fff8255e tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:21.823338 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-42bbf9df-37f1-4d0b-81de-61c4fff8255e tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:21.823769 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-42bbf9df-37f1-4d0b-81de-61c4fff8255e tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:21.824488 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-42bbf9df-37f1-4d0b-81de-61c4fff8255e tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:21.824940 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-42bbf9df-37f1-4d0b-81de-61c4fff8255e tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:21.825360 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-42bbf9df-37f1-4d0b-81de-61c4fff8255e tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:21.961297 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-42bbf9df-37f1-4d0b-81de-61c4fff8255e tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:21.969702 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-42bbf9df-37f1-4d0b-81de-61c4fff8255e tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1664 microversion: 2.1 time: 0.165564 Oct 30 01:03:21.970275 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 426/852] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:21 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1664 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:22.696794 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-a0d528c9-3975-45c6-a03f-ea911f165c02 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:22.697406 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-a0d528c9-3975-45c6-a03f-ea911f165c02 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:22.704355 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a0d528c9-3975-45c6-a03f-ea911f165c02 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:22.704847 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a0d528c9-3975-45c6-a03f-ea911f165c02 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:22.705282 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a0d528c9-3975-45c6-a03f-ea911f165c02 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:22.706001 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a0d528c9-3975-45c6-a03f-ea911f165c02 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:22.706454 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a0d528c9-3975-45c6-a03f-ea911f165c02 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:22.706910 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a0d528c9-3975-45c6-a03f-ea911f165c02 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:22.836542 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-a0d528c9-3975-45c6-a03f-ea911f165c02 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '05eed70effab4e38ae469ab44d1114db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dcbf2e9fb2e4a4698633fcbb07c30cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:22.844706 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-a0d528c9-3975-45c6-a03f-ea911f165c02 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7" status: 200 len: 1358 microversion: 2.1 time: 0.151859 Oct 30 01:03:22.845286 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 427/853] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:22 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7 => generated 1358 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:22.992724 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-b279702b-e507-4d16-b87e-05b12bc6f1c2 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:22.994312 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-b279702b-e507-4d16-b87e-05b12bc6f1c2 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:23.000183 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_api:SELECT=62,nova_api:INSERT=3 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:03:23.002069 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b279702b-e507-4d16-b87e-05b12bc6f1c2 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:23.002790 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b279702b-e507-4d16-b87e-05b12bc6f1c2 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:23.003392 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b279702b-e507-4d16-b87e-05b12bc6f1c2 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:23.004588 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b279702b-e507-4d16-b87e-05b12bc6f1c2 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:23.005259 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b279702b-e507-4d16-b87e-05b12bc6f1c2 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:23.005866 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b279702b-e507-4d16-b87e-05b12bc6f1c2 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:23.148843 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-b279702b-e507-4d16-b87e-05b12bc6f1c2 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:23.159941 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-b279702b-e507-4d16-b87e-05b12bc6f1c2 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1664 microversion: 2.1 time: 0.171067 Oct 30 01:03:23.163756 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 427/854] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:22 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1664 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:23.868085 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-eac2070a-096c-4f25-aaac-dd43bfeddd9b tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:23.868673 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-eac2070a-096c-4f25-aaac-dd43bfeddd9b tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:23.875677 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-eac2070a-096c-4f25-aaac-dd43bfeddd9b tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:23.876202 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-eac2070a-096c-4f25-aaac-dd43bfeddd9b tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:23.876635 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-eac2070a-096c-4f25-aaac-dd43bfeddd9b tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:23.877343 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-eac2070a-096c-4f25-aaac-dd43bfeddd9b tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:23.877792 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-eac2070a-096c-4f25-aaac-dd43bfeddd9b tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:23.878216 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-eac2070a-096c-4f25-aaac-dd43bfeddd9b tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:24.008597 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-eac2070a-096c-4f25-aaac-dd43bfeddd9b tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '05eed70effab4e38ae469ab44d1114db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dcbf2e9fb2e4a4698633fcbb07c30cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:24.017278 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-eac2070a-096c-4f25-aaac-dd43bfeddd9b tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7" status: 200 len: 1358 microversion: 2.1 time: 0.153193 Oct 30 01:03:24.017850 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 428/855] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:23 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7 => generated 1358 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:24.183790 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-409234a0-90ea-4411-844d-b353d295312c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:24.185027 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-409234a0-90ea-4411-844d-b353d295312c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:24.191153 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-409234a0-90ea-4411-844d-b353d295312c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:24.191637 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-409234a0-90ea-4411-844d-b353d295312c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:24.192108 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-409234a0-90ea-4411-844d-b353d295312c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:24.192801 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-409234a0-90ea-4411-844d-b353d295312c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:24.193254 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-409234a0-90ea-4411-844d-b353d295312c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:24.193675 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-409234a0-90ea-4411-844d-b353d295312c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:24.333244 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-409234a0-90ea-4411-844d-b353d295312c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:24.341644 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-409234a0-90ea-4411-844d-b353d295312c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1668 microversion: 2.1 time: 0.162352 Oct 30 01:03:24.342404 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 428/856] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:24 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1668 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 01:03:25.044967 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-078f6e36-7ae5-43f4-a5b3-6bbf62b7e7c6 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:25.045960 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-078f6e36-7ae5-43f4-a5b3-6bbf62b7e7c6 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:25.049936 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_api:SELECT=64,nova_api:INSERT=1 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:03:25.058502 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-078f6e36-7ae5-43f4-a5b3-6bbf62b7e7c6 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:25.058991 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-078f6e36-7ae5-43f4-a5b3-6bbf62b7e7c6 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:25.059422 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-078f6e36-7ae5-43f4-a5b3-6bbf62b7e7c6 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:25.060134 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-078f6e36-7ae5-43f4-a5b3-6bbf62b7e7c6 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:25.060581 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-078f6e36-7ae5-43f4-a5b3-6bbf62b7e7c6 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:25.060999 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-078f6e36-7ae5-43f4-a5b3-6bbf62b7e7c6 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:25.201319 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-078f6e36-7ae5-43f4-a5b3-6bbf62b7e7c6 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '05eed70effab4e38ae469ab44d1114db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dcbf2e9fb2e4a4698633fcbb07c30cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:25.209432 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-078f6e36-7ae5-43f4-a5b3-6bbf62b7e7c6 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7" status: 200 len: 1358 microversion: 2.1 time: 0.171148 Oct 30 01:03:25.210005 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 429/857] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:25 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7 => generated 1358 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:25.265759 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [req-aaa4a316-1098-4afd-97fd-c94ddb584cbf req-24bb9132-95f9-441a-bb09-66be58e4dd46 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6f5485fd-b825-45e7-a5be-eb96208c09b7", "name": "network-vif-plugged", "status": "completed", "tag": "ad10e942-b527-4fbd-867f-e161a78a11bc"}]} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:03:25.278838 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-aaa4a316-1098-4afd-97fd-c94ddb584cbf req-24bb9132-95f9-441a-bb09-66be58e4dd46 service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:25.279328 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-aaa4a316-1098-4afd-97fd-c94ddb584cbf req-24bb9132-95f9-441a-bb09-66be58e4dd46 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:25.279856 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-aaa4a316-1098-4afd-97fd-c94ddb584cbf req-24bb9132-95f9-441a-bb09-66be58e4dd46 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:25.292382 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.compute.server_external_events [req-aaa4a316-1098-4afd-97fd-c94ddb584cbf req-24bb9132-95f9-441a-bb09-66be58e4dd46 service nova] Creating event network-vif-plugged:ad10e942-b527-4fbd-867f-e161a78a11bc for instance 6f5485fd-b825-45e7-a5be-eb96208c09b7 on hv-cinder-maste Oct 30 01:03:25.297380 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [req-aaa4a316-1098-4afd-97fd-c94ddb584cbf req-24bb9132-95f9-441a-bb09-66be58e4dd46 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037144 Oct 30 01:03:25.298177 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 429/858] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:03:25 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:03:26.235546 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-0b0fffc9-1279-435f-b37f-668a53981f7f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:26.236165 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-0b0fffc9-1279-435f-b37f-668a53981f7f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:26.244812 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0b0fffc9-1279-435f-b37f-668a53981f7f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:26.245482 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0b0fffc9-1279-435f-b37f-668a53981f7f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:26.246133 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0b0fffc9-1279-435f-b37f-668a53981f7f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:26.246999 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0b0fffc9-1279-435f-b37f-668a53981f7f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:26.247565 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0b0fffc9-1279-435f-b37f-668a53981f7f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:26.248158 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0b0fffc9-1279-435f-b37f-668a53981f7f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:26.381005 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-0b0fffc9-1279-435f-b37f-668a53981f7f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '05eed70effab4e38ae469ab44d1114db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dcbf2e9fb2e4a4698633fcbb07c30cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:26.389004 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-0b0fffc9-1279-435f-b37f-668a53981f7f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7" status: 200 len: 1358 microversion: 2.1 time: 0.158675 Oct 30 01:03:26.389544 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 430/859] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:26 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7 => generated 1358 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:27.418563 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-56d64ea8-4cdd-49b1-b7ce-018d9d3a6567 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:27.419760 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-56d64ea8-4cdd-49b1-b7ce-018d9d3a6567 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:27.430149 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-56d64ea8-4cdd-49b1-b7ce-018d9d3a6567 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:27.430953 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-56d64ea8-4cdd-49b1-b7ce-018d9d3a6567 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:27.431673 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-56d64ea8-4cdd-49b1-b7ce-018d9d3a6567 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:27.433077 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-56d64ea8-4cdd-49b1-b7ce-018d9d3a6567 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:27.433559 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-56d64ea8-4cdd-49b1-b7ce-018d9d3a6567 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:27.433977 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-56d64ea8-4cdd-49b1-b7ce-018d9d3a6567 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:27.581389 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-56d64ea8-4cdd-49b1-b7ce-018d9d3a6567 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '05eed70effab4e38ae469ab44d1114db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dcbf2e9fb2e4a4698633fcbb07c30cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:27.593841 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-56d64ea8-4cdd-49b1-b7ce-018d9d3a6567 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7" status: 200 len: 1358 microversion: 2.1 time: 0.183357 Oct 30 01:03:27.594606 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 430/860] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:27 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7 => generated 1358 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:28.624716 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-1367ff4f-2980-430a-960b-47cbe966e06b tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:28.625571 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-1367ff4f-2980-430a-960b-47cbe966e06b tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:28.636607 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1367ff4f-2980-430a-960b-47cbe966e06b tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:28.637093 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1367ff4f-2980-430a-960b-47cbe966e06b tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:28.637527 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1367ff4f-2980-430a-960b-47cbe966e06b tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:28.638224 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1367ff4f-2980-430a-960b-47cbe966e06b tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:28.638672 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1367ff4f-2980-430a-960b-47cbe966e06b tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:28.639092 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1367ff4f-2980-430a-960b-47cbe966e06b tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:28.771261 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-1367ff4f-2980-430a-960b-47cbe966e06b tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '05eed70effab4e38ae469ab44d1114db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dcbf2e9fb2e4a4698633fcbb07c30cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:28.780801 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-1367ff4f-2980-430a-960b-47cbe966e06b tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7" status: 200 len: 1549 microversion: 2.1 time: 0.162806 Oct 30 01:03:28.781375 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 431/861] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:28 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7 => generated 1549 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:28.803467 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-466462f5-bd6f-49f1-b452-e4cdf4519790 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:28.804086 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-466462f5-bd6f-49f1-b452-e4cdf4519790 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:28.810810 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-466462f5-bd6f-49f1-b452-e4cdf4519790 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:28.811285 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-466462f5-bd6f-49f1-b452-e4cdf4519790 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:28.811707 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-466462f5-bd6f-49f1-b452-e4cdf4519790 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:28.812430 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-466462f5-bd6f-49f1-b452-e4cdf4519790 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:28.812870 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-466462f5-bd6f-49f1-b452-e4cdf4519790 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:28.813284 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-466462f5-bd6f-49f1-b452-e4cdf4519790 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:28.943499 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-466462f5-bd6f-49f1-b452-e4cdf4519790 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '05eed70effab4e38ae469ab44d1114db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dcbf2e9fb2e4a4698633fcbb07c30cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:28.951551 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-466462f5-bd6f-49f1-b452-e4cdf4519790 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7" status: 200 len: 1549 microversion: 2.1 time: 0.151911 Oct 30 01:03:28.952152 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 431/862] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:28 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7 => generated 1549 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:28.976995 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-565ad5bd-858b-4fd1-94a9-b6dd6b2ef2f1 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:28.985184 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-565ad5bd-858b-4fd1-94a9-b6dd6b2ef2f1 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '05eed70effab4e38ae469ab44d1114db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dcbf2e9fb2e4a4698633fcbb07c30cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:28.986395 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-565ad5bd-858b-4fd1-94a9-b6dd6b2ef2f1 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Searching by: {'deleted': False, 'project_id': '1dcbf2e9fb2e4a4698633fcbb07c30cd'} {{(pid=118962) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 30 01:03:28.996417 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-565ad5bd-858b-4fd1-94a9-b6dd6b2ef2f1 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:28.996898 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-565ad5bd-858b-4fd1-94a9-b6dd6b2ef2f1 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:28.997336 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-565ad5bd-858b-4fd1-94a9-b6dd6b2ef2f1 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:28.998059 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-565ad5bd-858b-4fd1-94a9-b6dd6b2ef2f1 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:28.998529 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-565ad5bd-858b-4fd1-94a9-b6dd6b2ef2f1 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:28.998924 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-565ad5bd-858b-4fd1-94a9-b6dd6b2ef2f1 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:29.089633 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.multi_cell_list [None req-565ad5bd-858b-4fd1-94a9-b6dd6b2ef2f1 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=118962) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Oct 30 01:03:29.182909 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-565ad5bd-858b-4fd1-94a9-b6dd6b2ef2f1 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers" status: 200 len: 342 microversion: 2.1 time: 0.211561 Oct 30 01:03:29.183601 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 432/863] 10.222.0.16 () {58 vars in 1125 bytes} [Sun Oct 30 01:03:28 2022] GET /compute/v2.1/servers => generated 342 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:03:29.198612 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-2970b5c0-a55c-4f23-84df-4616f0cdb049 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:29.199301 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-2970b5c0-a55c-4f23-84df-4616f0cdb049 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:29.205611 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2970b5c0-a55c-4f23-84df-4616f0cdb049 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:29.206091 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2970b5c0-a55c-4f23-84df-4616f0cdb049 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:29.206517 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2970b5c0-a55c-4f23-84df-4616f0cdb049 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:29.207206 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2970b5c0-a55c-4f23-84df-4616f0cdb049 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:29.207997 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2970b5c0-a55c-4f23-84df-4616f0cdb049 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:29.208414 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2970b5c0-a55c-4f23-84df-4616f0cdb049 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:29.342688 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-2970b5c0-a55c-4f23-84df-4616f0cdb049 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '05eed70effab4e38ae469ab44d1114db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dcbf2e9fb2e4a4698633fcbb07c30cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:29.351283 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-2970b5c0-a55c-4f23-84df-4616f0cdb049 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7" status: 200 len: 1549 microversion: 2.1 time: 0.156588 Oct 30 01:03:29.352007 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 432/864] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:29 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7 => generated 1549 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:30.943915 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-ff56796c-bc35-4ab0-8187-a4dbe19b178c tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "a125d045-591a-4811-a456-a51a7989295e"}} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:03:30.947666 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-ff56796c-bc35-4ab0-8187-a4dbe19b178c tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:30.958135 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ff56796c-bc35-4ab0-8187-a4dbe19b178c tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:30.958847 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ff56796c-bc35-4ab0-8187-a4dbe19b178c tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:30.959525 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ff56796c-bc35-4ab0-8187-a4dbe19b178c tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:30.960653 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ff56796c-bc35-4ab0-8187-a4dbe19b178c tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:30.961333 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ff56796c-bc35-4ab0-8187-a4dbe19b178c tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:30.961956 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ff56796c-bc35-4ab0-8187-a4dbe19b178c tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:31.263287 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-ff56796c-bc35-4ab0-8187-a4dbe19b178c tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "POST /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.326159 Oct 30 01:03:31.263966 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 433/865] 10.222.0.16 () {60 vars in 1323 bytes} [Sun Oct 30 01:03:30 2022] POST /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments => generated 194 bytes in 328 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:03:38.943147 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-44d5ebea-19b5-4f82-88bd-b44717db11a6 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:38.943745 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-44d5ebea-19b5-4f82-88bd-b44717db11a6 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:38.950471 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-44d5ebea-19b5-4f82-88bd-b44717db11a6 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:38.950951 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-44d5ebea-19b5-4f82-88bd-b44717db11a6 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:38.951376 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-44d5ebea-19b5-4f82-88bd-b44717db11a6 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:38.952092 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-44d5ebea-19b5-4f82-88bd-b44717db11a6 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:38.952534 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-44d5ebea-19b5-4f82-88bd-b44717db11a6 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:38.952945 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-44d5ebea-19b5-4f82-88bd-b44717db11a6 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:39.083690 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-44d5ebea-19b5-4f82-88bd-b44717db11a6 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '05eed70effab4e38ae469ab44d1114db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dcbf2e9fb2e4a4698633fcbb07c30cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:39.092661 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-44d5ebea-19b5-4f82-88bd-b44717db11a6 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7" status: 200 len: 1595 microversion: 2.1 time: 0.153436 Oct 30 01:03:39.093244 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 433/866] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:38 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7 => generated 1595 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:39.723032 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell0:SELECT=2 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:03:39.727480 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-769984d3-4855-4bad-9966-148ba9cd5628 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:39.728098 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-769984d3-4855-4bad-9966-148ba9cd5628 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5632075c-7062-434e-b00c-bbf04c5ac04a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:39.740354 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-769984d3-4855-4bad-9966-148ba9cd5628 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:39.740836 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-769984d3-4855-4bad-9966-148ba9cd5628 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:39.741259 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-769984d3-4855-4bad-9966-148ba9cd5628 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:39.741954 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-769984d3-4855-4bad-9966-148ba9cd5628 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:39.742397 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-769984d3-4855-4bad-9966-148ba9cd5628 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:39.742810 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-769984d3-4855-4bad-9966-148ba9cd5628 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:39.764113 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:03:39.770806 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-769984d3-4855-4bad-9966-148ba9cd5628 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a/ips" status: 200 len: 150 microversion: 2.1 time: 0.049482 Oct 30 01:03:39.771394 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 434/867] 10.222.0.16 () {58 vars in 1248 bytes} [Sun Oct 30 01:03:39 2022] GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a/ips => generated 150 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:03:39.783330 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-dc037f80-d7b5-48d9-921c-3b81083bbb94 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "00c66df2-3c81-4adf-8028-8c47eb8b88ba"}} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:03:39.786022 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-dc037f80-d7b5-48d9-921c-3b81083bbb94 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5632075c-7062-434e-b00c-bbf04c5ac04a] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:39.792401 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-dc037f80-d7b5-48d9-921c-3b81083bbb94 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:39.792882 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-dc037f80-d7b5-48d9-921c-3b81083bbb94 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:39.793341 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-dc037f80-d7b5-48d9-921c-3b81083bbb94 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:39.794027 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-dc037f80-d7b5-48d9-921c-3b81083bbb94 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:39.794472 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-dc037f80-d7b5-48d9-921c-3b81083bbb94 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:39.794895 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-dc037f80-d7b5-48d9-921c-3b81083bbb94 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:39.877796 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.wsgi [None req-dc037f80-d7b5-48d9-921c-3b81083bbb94 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] HTTP exception thrown: Invalid volume: volume 00c66df2-3c81-4adf-8028-8c47eb8b88ba is already attached to instances: 5652be64-c2a3-4b24-a3aa-c962f204c06e Oct 30 01:03:39.879562 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-dc037f80-d7b5-48d9-921c-3b81083bbb94 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Request method failure captured: Oct 30 01:03:39.879562 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: request: POST /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a/os-volume_attachments HTTP/1.1 Oct 30 01:03:39.879562 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Accept: application/json Oct 30 01:03:39.879562 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Accept-Encoding: identity Oct 30 01:03:39.879562 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Connection: close Oct 30 01:03:39.879562 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Content-Length: 74 Oct 30 01:03:39.879562 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Content-Type: application/json Oct 30 01:03:39.879562 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Host: 10.222.0.16 Oct 30 01:03:39.879562 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Openstack-System-Scope: None Oct 30 01:03:39.879562 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: User-Agent: python-urllib3/1.26.12 Oct 30 01:03:39.879562 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Auth-Token: gAAAAABjXcuedsaRjtnPRcHo4HqTnifOYe1vcHHjSzl7t8SsEKfOt41JBob3t9Ien606ucEZmcgmpf5H0lzlL6oZS4vLpZERLur9y2YRhqd9AYPyhbrmUkrYmKAssv6PSCysf6z3-myn1YKr62Nxqfs7t1lZGZnZuH_I2j8LHl5PnVXWJxyADCY Oct 30 01:03:39.879562 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Domain-Id: None Oct 30 01:03:39.879562 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Domain-Name: None Oct 30 01:03:39.879562 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Identity-Status: Confirmed Oct 30 01:03:39.879562 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Is-Admin-Project: True Oct 30 01:03:39.879562 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Domain-Id: default Oct 30 01:03:39.879562 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Domain-Name: Default Oct 30 01:03:39.879562 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Id: 6b68b886c19c45d585409651f33b8626 Oct 30 01:03:39.879562 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Name: tempest-AttachVolumeNegativeTest-618009204 Oct 30 01:03:39.879562 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Role: reader,member Oct 30 01:03:39.879562 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Roles: reader,member Oct 30 01:03:39.881324 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_6b68b886c19c45d585409651f33b8626"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6b68b886c19c45d585409651f33b8626"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/6b68b886c19c45d585409651f33b8626"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6b68b886c19c45d585409651f33b8626"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 01:03:39.881324 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant: tempest-AttachVolumeNegativeTest-618009204 Oct 30 01:03:39.881324 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant-Id: 6b68b886c19c45d585409651f33b8626 Oct 30 01:03:39.881324 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-618009204 Oct 30 01:03:39.881324 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User: tempest-AttachVolumeNegativeTest-618009204-project Oct 30 01:03:39.881324 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Domain-Id: default Oct 30 01:03:39.881324 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Domain-Name: Default Oct 30 01:03:39.881324 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Id: ecaf8ec105c04b919256f852da859eba Oct 30 01:03:39.881324 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Name: tempest-AttachVolumeNegativeTest-618009204-project Oct 30 01:03:39.881324 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Oct 30 01:03:39.881324 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: {"volumeAttachment": {"volumeId": "00c66df2-3c81-4adf-8028-8c47eb8b88ba"}} Oct 30 01:03:39.881324 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: method: > Oct 30 01:03:39.882095 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: exception: Invalid volume: volume 00c66df2-3c81-4adf-8028-8c47eb8b88ba is already attached to instances: 5652be64-c2a3-4b24-a3aa-c962f204c06e Oct 30 01:03:39.882095 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: action_args: {'server_id': '5632075c-7062-434e-b00c-bbf04c5ac04a', 'body': {'volumeAttachment': {'volumeId': '00c66df2-3c81-4adf-8028-8c47eb8b88ba'}}} Oct 30 01:03:39.882095 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 01:03:39.882095 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:03:39.882095 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Oct 30 01:03:39.882095 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Oct 30 01:03:39.882095 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 30 01:03:39.882095 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 30 01:03:39.882095 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Oct 30 01:03:39.882095 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Oct 30 01:03:39.882095 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5130, in attach_volume Oct 30 01:03:39.882095 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Oct 30 01:03:39.882095 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4970, in _check_volume_already_attached Oct 30 01:03:39.882095 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Oct 30 01:03:39.882095 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume 00c66df2-3c81-4adf-8028-8c47eb8b88ba is already attached to instances: 5652be64-c2a3-4b24-a3aa-c962f204c06e Oct 30 01:03:39.882095 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:03:39.882095 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:03:39.882095 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:03:39.882095 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:03:39.883251 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:03:39.883251 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:03:39.883251 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 01:03:39.883251 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 01:03:39.883251 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 01:03:39.883251 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 01:03:39.883251 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 30 01:03:39.883251 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 30 01:03:39.883251 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 30 01:03:39.883251 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 30 01:03:39.883251 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 30 01:03:39.883251 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 30 01:03:39.883251 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Oct 30 01:03:39.883251 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Oct 30 01:03:39.883251 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume 00c66df2-3c81-4adf-8028-8c47eb8b88ba is already attached to instances: 5652be64-c2a3-4b24-a3aa-c962f204c06e Oct 30 01:03:39.883251 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:03:39.883251 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:03:39.883251 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:03:39.883251 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:03:39.883251 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:03:39.884477 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:03:39.884477 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 01:03:39.884477 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 01:03:39.884477 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume 00c66df2-3c81-4adf-8028-8c47eb8b88ba is already attached to instances: 5652be64-c2a3-4b24-a3aa-c962f204c06e Oct 30 01:03:39.884477 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:03:39.884477 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-dc037f80-d7b5-48d9-921c-3b81083bbb94 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Returning 400 to user: Invalid volume: volume 00c66df2-3c81-4adf-8028-8c47eb8b88ba is already attached to instances: 5652be64-c2a3-4b24-a3aa-c962f204c06e {{(pid=118961) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 01:03:39.884477 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-dc037f80-d7b5-48d9-921c-3b81083bbb94 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "POST /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.101825 Oct 30 01:03:39.884477 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 434/868] 10.222.0.16 () {60 vars in 1323 bytes} [Sun Oct 30 01:03:39 2022] POST /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a/os-volume_attachments => generated 174 bytes in 103 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 30 01:03:39.890258 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-a23619ac-9bd0-4f97-81e9-75bb34feb457 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:39.890753 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-a23619ac-9bd0-4f97-81e9-75bb34feb457 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5632075c-7062-434e-b00c-bbf04c5ac04a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:39.898379 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a23619ac-9bd0-4f97-81e9-75bb34feb457 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:39.898854 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a23619ac-9bd0-4f97-81e9-75bb34feb457 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:39.899278 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a23619ac-9bd0-4f97-81e9-75bb34feb457 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:39.899994 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a23619ac-9bd0-4f97-81e9-75bb34feb457 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:39.900437 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a23619ac-9bd0-4f97-81e9-75bb34feb457 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:39.900851 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a23619ac-9bd0-4f97-81e9-75bb34feb457 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:39.927230 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-a23619ac-9bd0-4f97-81e9-75bb34feb457 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5632075c-7062-434e-b00c-bbf04c5ac04a] Going to try to terminate instance {{(pid=118962) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 30 01:03:40.004922 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-a23619ac-9bd0-4f97-81e9-75bb34feb457 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "DELETE /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a" status: 204 len: 0 microversion: 2.1 time: 0.118177 Oct 30 01:03:40.008728 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 435/869] 10.222.0.16 () {58 vars in 1239 bytes} [Sun Oct 30 01:03:39 2022] DELETE /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a => generated 0 bytes in 120 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 30 01:03:40.014375 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-44171058-24ae-4040-b755-5276ee162115 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:40.014998 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-44171058-24ae-4040-b755-5276ee162115 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5632075c-7062-434e-b00c-bbf04c5ac04a] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:40.023757 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-44171058-24ae-4040-b755-5276ee162115 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:40.024255 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-44171058-24ae-4040-b755-5276ee162115 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:40.024695 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-44171058-24ae-4040-b755-5276ee162115 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:40.025405 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-44171058-24ae-4040-b755-5276ee162115 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:40.025859 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-44171058-24ae-4040-b755-5276ee162115 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:40.026284 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-44171058-24ae-4040-b755-5276ee162115 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:40.236784 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-44171058-24ae-4040-b755-5276ee162115 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:40.245373 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-44171058-24ae-4040-b755-5276ee162115 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a" status: 200 len: 1681 microversion: 2.1 time: 0.235088 Oct 30 01:03:40.245946 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 435/870] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:40 2022] GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a => generated 1681 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:41.137520 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [req-c968b9e3-3997-4f16-8ebc-07c6795cb20b req-1a7fe671-71ee-4118-9806-15594aea9b5e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6f5485fd-b825-45e7-a5be-eb96208c09b7", "tag": "ad10e942-b527-4fbd-867f-e161a78a11bc"}]} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:03:41.148311 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-c968b9e3-3997-4f16-8ebc-07c6795cb20b req-1a7fe671-71ee-4118-9806-15594aea9b5e service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:41.148790 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-c968b9e3-3997-4f16-8ebc-07c6795cb20b req-1a7fe671-71ee-4118-9806-15594aea9b5e service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:41.149203 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-c968b9e3-3997-4f16-8ebc-07c6795cb20b req-1a7fe671-71ee-4118-9806-15594aea9b5e service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:41.160965 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.compute.server_external_events [req-c968b9e3-3997-4f16-8ebc-07c6795cb20b req-1a7fe671-71ee-4118-9806-15594aea9b5e service nova] Creating event network-changed:ad10e942-b527-4fbd-867f-e161a78a11bc for instance 6f5485fd-b825-45e7-a5be-eb96208c09b7 on hv-cinder-maste Oct 30 01:03:41.165998 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [req-c968b9e3-3997-4f16-8ebc-07c6795cb20b req-1a7fe671-71ee-4118-9806-15594aea9b5e service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032445 Oct 30 01:03:41.166605 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 436/871] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:03:41 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:03:41.269915 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-a3ae785b-47aa-4fc7-88fe-c7cd9de5ed8f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:41.270565 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-a3ae785b-47aa-4fc7-88fe-c7cd9de5ed8f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5632075c-7062-434e-b00c-bbf04c5ac04a] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:41.276919 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a3ae785b-47aa-4fc7-88fe-c7cd9de5ed8f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:41.277395 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a3ae785b-47aa-4fc7-88fe-c7cd9de5ed8f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:41.278039 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a3ae785b-47aa-4fc7-88fe-c7cd9de5ed8f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:41.278817 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a3ae785b-47aa-4fc7-88fe-c7cd9de5ed8f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:41.279823 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a3ae785b-47aa-4fc7-88fe-c7cd9de5ed8f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:41.280470 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a3ae785b-47aa-4fc7-88fe-c7cd9de5ed8f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:41.456596 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-a3ae785b-47aa-4fc7-88fe-c7cd9de5ed8f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:41.465178 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-a3ae785b-47aa-4fc7-88fe-c7cd9de5ed8f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a" status: 200 len: 1681 microversion: 2.1 time: 0.199645 Oct 30 01:03:41.466335 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 436/872] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:41 2022] GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a => generated 1681 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:42.235944 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-88960c13-87d2-4e85-bc94-feb15ac3cf25 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-secgroup-smoke-1066949310"}} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:03:42.236558 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-88960c13-87d2-4e85-bc94-feb15ac3cf25 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:42.242664 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-88960c13-87d2-4e85-bc94-feb15ac3cf25 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:42.243138 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-88960c13-87d2-4e85-bc94-feb15ac3cf25 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:42.243563 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-88960c13-87d2-4e85-bc94-feb15ac3cf25 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:42.244277 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-88960c13-87d2-4e85-bc94-feb15ac3cf25 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:42.244716 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-88960c13-87d2-4e85-bc94-feb15ac3cf25 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:42.245130 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-88960c13-87d2-4e85-bc94-feb15ac3cf25 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:42.381296 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.network.security_group_api [None req-88960c13-87d2-4e85-bc94-feb15ac3cf25 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Adding security group 2f032d87-6dca-4e2a-afc8-c3c7a20bc0ff to port ad10e942-b527-4fbd-867f-e161a78a11bc Oct 30 01:03:42.490587 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-0e64946e-5728-4828-8f4a-572f6c8fe893 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:42.491297 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-0e64946e-5728-4828-8f4a-572f6c8fe893 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5632075c-7062-434e-b00c-bbf04c5ac04a] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:42.497734 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0e64946e-5728-4828-8f4a-572f6c8fe893 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:42.498411 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0e64946e-5728-4828-8f4a-572f6c8fe893 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:42.498990 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0e64946e-5728-4828-8f4a-572f6c8fe893 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:42.499932 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0e64946e-5728-4828-8f4a-572f6c8fe893 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:42.500530 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0e64946e-5728-4828-8f4a-572f6c8fe893 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:42.501085 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0e64946e-5728-4828-8f4a-572f6c8fe893 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:42.520075 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell1:SELECT=100 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:03:42.728085 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-0e64946e-5728-4828-8f4a-572f6c8fe893 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:42.736993 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-0e64946e-5728-4828-8f4a-572f6c8fe893 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a" status: 200 len: 1681 microversion: 2.1 time: 0.250250 Oct 30 01:03:42.738400 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 437/873] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:42 2022] GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a => generated 1681 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 01:03:42.793677 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-88960c13-87d2-4e85-bc94-feb15ac3cf25 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "POST /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/action" status: 202 len: 0 microversion: 2.1 time: 0.562414 Oct 30 01:03:42.794584 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 437/874] 10.222.0.16 () {60 vars in 1278 bytes} [Sun Oct 30 01:03:42 2022] POST /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/action => generated 0 bytes in 564 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 30 01:03:42.803067 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-a9ed61c0-470b-4638-88ea-3562f83dfd00 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:42.803739 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-a9ed61c0-470b-4638-88ea-3562f83dfd00 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:42.810010 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a9ed61c0-470b-4638-88ea-3562f83dfd00 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:42.810632 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a9ed61c0-470b-4638-88ea-3562f83dfd00 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:42.811047 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a9ed61c0-470b-4638-88ea-3562f83dfd00 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:42.812004 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a9ed61c0-470b-4638-88ea-3562f83dfd00 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:42.812451 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a9ed61c0-470b-4638-88ea-3562f83dfd00 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:42.812858 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a9ed61c0-470b-4638-88ea-3562f83dfd00 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:42.997587 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-a9ed61c0-470b-4638-88ea-3562f83dfd00 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '05eed70effab4e38ae469ab44d1114db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dcbf2e9fb2e4a4698633fcbb07c30cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:43.007561 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-a9ed61c0-470b-4638-88ea-3562f83dfd00 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7" status: 200 len: 1642 microversion: 2.1 time: 0.208331 Oct 30 01:03:43.008887 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 438/875] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:42 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7 => generated 1642 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:43.182999 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [req-1d9245b0-1c36-423d-958f-9615fff4a93c req-3d0b611c-12b9-4874-a4a5-7b173c8e2c68 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6f5485fd-b825-45e7-a5be-eb96208c09b7", "tag": "ad10e942-b527-4fbd-867f-e161a78a11bc"}]} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:03:43.194107 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-1d9245b0-1c36-423d-958f-9615fff4a93c req-3d0b611c-12b9-4874-a4a5-7b173c8e2c68 service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:43.197539 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-1d9245b0-1c36-423d-958f-9615fff4a93c req-3d0b611c-12b9-4874-a4a5-7b173c8e2c68 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:43.197969 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-1d9245b0-1c36-423d-958f-9615fff4a93c req-3d0b611c-12b9-4874-a4a5-7b173c8e2c68 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:43.213276 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.compute.server_external_events [req-1d9245b0-1c36-423d-958f-9615fff4a93c req-3d0b611c-12b9-4874-a4a5-7b173c8e2c68 service nova] Creating event network-changed:ad10e942-b527-4fbd-867f-e161a78a11bc for instance 6f5485fd-b825-45e7-a5be-eb96208c09b7 on hv-cinder-maste Oct 30 01:03:43.219977 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [req-1d9245b0-1c36-423d-958f-9615fff4a93c req-3d0b611c-12b9-4874-a4a5-7b173c8e2c68 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.041423 Oct 30 01:03:43.220909 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 438/876] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:03:43 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:03:43.762571 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-3bc18d03-3852-4848-aa42-57afb85013d7 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:43.764092 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-3bc18d03-3852-4848-aa42-57afb85013d7 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5632075c-7062-434e-b00c-bbf04c5ac04a] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:43.770812 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3bc18d03-3852-4848-aa42-57afb85013d7 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:43.771586 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3bc18d03-3852-4848-aa42-57afb85013d7 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:43.772209 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3bc18d03-3852-4848-aa42-57afb85013d7 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:43.773091 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3bc18d03-3852-4848-aa42-57afb85013d7 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:43.773740 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3bc18d03-3852-4848-aa42-57afb85013d7 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:43.774323 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3bc18d03-3852-4848-aa42-57afb85013d7 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:43.819239 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [req-419a410c-f59c-46f0-b594-5fbc11c8d1b6 req-927dfaf6-82d4-4249-8e35-1ebda11729b1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5632075c-7062-434e-b00c-bbf04c5ac04a", "name": "network-vif-deleted", "tag": "c9fe8633-400b-4adf-99ff-00c04190ea67"}]} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:03:43.840276 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-419a410c-f59c-46f0-b594-5fbc11c8d1b6 req-927dfaf6-82d4-4249-8e35-1ebda11729b1 service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:43.841715 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-419a410c-f59c-46f0-b594-5fbc11c8d1b6 req-927dfaf6-82d4-4249-8e35-1ebda11729b1 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:43.842684 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-419a410c-f59c-46f0-b594-5fbc11c8d1b6 req-927dfaf6-82d4-4249-8e35-1ebda11729b1 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:43.863383 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.compute.server_external_events [req-419a410c-f59c-46f0-b594-5fbc11c8d1b6 req-927dfaf6-82d4-4249-8e35-1ebda11729b1 service nova] Creating event network-vif-deleted:c9fe8633-400b-4adf-99ff-00c04190ea67 for instance 5632075c-7062-434e-b00c-bbf04c5ac04a on hv-cinder-maste Oct 30 01:03:43.874245 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [req-419a410c-f59c-46f0-b594-5fbc11c8d1b6 req-927dfaf6-82d4-4249-8e35-1ebda11729b1 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.063678 Oct 30 01:03:43.877990 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 439/877] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:03:43 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 30 01:03:43.913585 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-3bc18d03-3852-4848-aa42-57afb85013d7 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:43.927327 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-3bc18d03-3852-4848-aa42-57afb85013d7 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a" status: 200 len: 1613 microversion: 2.1 time: 0.168587 Oct 30 01:03:43.935680 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 439/878] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:43 2022] GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a => generated 1613 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:44.964853 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-1d20208f-cb04-4b28-abf1-38954365db87 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:44.965596 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-1d20208f-cb04-4b28-abf1-38954365db87 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5632075c-7062-434e-b00c-bbf04c5ac04a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:44.972365 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1d20208f-cb04-4b28-abf1-38954365db87 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:44.972833 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1d20208f-cb04-4b28-abf1-38954365db87 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:44.973254 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1d20208f-cb04-4b28-abf1-38954365db87 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:44.973963 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1d20208f-cb04-4b28-abf1-38954365db87 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:44.974403 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1d20208f-cb04-4b28-abf1-38954365db87 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:44.974818 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1d20208f-cb04-4b28-abf1-38954365db87 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:44.989335 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.wsgi [None req-1d20208f-cb04-4b28-abf1-38954365db87 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] HTTP exception thrown: Instance 5632075c-7062-434e-b00c-bbf04c5ac04a could not be found. Oct 30 01:03:44.991014 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-1d20208f-cb04-4b28-abf1-38954365db87 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Request method failure captured: Oct 30 01:03:44.991014 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: request: GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a HTTP/1.1 Oct 30 01:03:44.991014 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept: application/json Oct 30 01:03:44.991014 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept-Encoding: identity Oct 30 01:03:44.991014 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Connection: close Oct 30 01:03:44.991014 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Length: 0 Oct 30 01:03:44.991014 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Type: application/json Oct 30 01:03:44.991014 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Host: 10.222.0.16 Oct 30 01:03:44.991014 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Openstack-System-Scope: None Oct 30 01:03:44.991014 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: User-Agent: python-urllib3/1.26.12 Oct 30 01:03:44.991014 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Auth-Token: gAAAAABjXcuedsaRjtnPRcHo4HqTnifOYe1vcHHjSzl7t8SsEKfOt41JBob3t9Ien606ucEZmcgmpf5H0lzlL6oZS4vLpZERLur9y2YRhqd9AYPyhbrmUkrYmKAssv6PSCysf6z3-myn1YKr62Nxqfs7t1lZGZnZuH_I2j8LHl5PnVXWJxyADCY Oct 30 01:03:44.991014 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Id: None Oct 30 01:03:44.991014 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Name: None Oct 30 01:03:44.991014 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Identity-Status: Confirmed Oct 30 01:03:44.991014 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Is-Admin-Project: True Oct 30 01:03:44.991014 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Id: default Oct 30 01:03:44.991014 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Name: Default Oct 30 01:03:44.991014 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Id: 6b68b886c19c45d585409651f33b8626 Oct 30 01:03:44.991014 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Name: tempest-AttachVolumeNegativeTest-618009204 Oct 30 01:03:44.991014 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Role: reader,member Oct 30 01:03:44.991014 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Roles: reader,member Oct 30 01:03:44.992664 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_6b68b886c19c45d585409651f33b8626"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6b68b886c19c45d585409651f33b8626"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/6b68b886c19c45d585409651f33b8626"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6b68b886c19c45d585409651f33b8626"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 01:03:44.992664 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant: tempest-AttachVolumeNegativeTest-618009204 Oct 30 01:03:44.992664 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Id: 6b68b886c19c45d585409651f33b8626 Oct 30 01:03:44.992664 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-618009204 Oct 30 01:03:44.992664 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User: tempest-AttachVolumeNegativeTest-618009204-project Oct 30 01:03:44.992664 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Id: default Oct 30 01:03:44.992664 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Name: Default Oct 30 01:03:44.992664 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Id: ecaf8ec105c04b919256f852da859eba Oct 30 01:03:44.992664 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Name: tempest-AttachVolumeNegativeTest-618009204-project Oct 30 01:03:44.992664 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: method: > Oct 30 01:03:44.992664 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: exception: Instance 5632075c-7062-434e-b00c-bbf04c5ac04a could not be found. Oct 30 01:03:44.992664 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: action_args: {'id': '5632075c-7062-434e-b00c-bbf04c5ac04a'} Oct 30 01:03:44.997168 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 01:03:44.997168 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:03:44.997168 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 30 01:03:44.997168 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 30 01:03:44.997168 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 30 01:03:44.997168 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 30 01:03:44.997168 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 30 01:03:44.997168 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 30 01:03:44.997168 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 30 01:03:44.997168 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 30 01:03:44.997168 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5632075c-7062-434e-b00c-bbf04c5ac04a could not be found. Oct 30 01:03:44.997168 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:03:44.997168 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:03:44.997168 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:03:44.997168 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:03:44.997168 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:03:44.997168 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:03:44.997168 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 01:03:44.997168 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 01:03:44.997168 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 01:03:44.998681 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 01:03:44.998681 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 30 01:03:44.998681 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 30 01:03:44.998681 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 30 01:03:44.998681 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 30 01:03:44.998681 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 30 01:03:44.998681 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 30 01:03:44.998681 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5632075c-7062-434e-b00c-bbf04c5ac04a could not be found. Oct 30 01:03:44.998681 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:03:44.998681 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:03:44.998681 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:03:44.998681 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:03:44.998681 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:03:44.998681 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:03:44.998681 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 01:03:44.998681 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 01:03:44.998681 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5632075c-7062-434e-b00c-bbf04c5ac04a could not be found. Oct 30 01:03:44.998681 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:03:45.000060 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-1d20208f-cb04-4b28-abf1-38954365db87 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Returning 404 to user: Instance 5632075c-7062-434e-b00c-bbf04c5ac04a could not be found. {{(pid=118962) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 01:03:45.000060 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-1d20208f-cb04-4b28-abf1-38954365db87 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a" status: 404 len: 111 microversion: 2.1 time: 0.033140 Oct 30 01:03:45.000060 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 440/879] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:44 2022] GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 30 01:03:45.002553 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-106da42b-5b9b-4df0-a9ec-bfb1b4fc164f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '.version_select at 0x7f0eab9e7160>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:45.025082 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-106da42b-5b9b-4df0-a9ec-bfb1b4fc164f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-457360226" status: 202 len: 0 microversion: 2.1 time: 0.026975 Oct 30 01:03:45.025770 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 440/880] 10.222.0.16 () {58 vars in 1218 bytes} [Sun Oct 30 01:03:44 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-457360226 => generated 0 bytes in 28 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 30 01:03:45.714536 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-c97722f9-5ceb-4b9f-869f-9cb116d7732b tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:45.715082 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-c97722f9-5ceb-4b9f-869f-9cb116d7732b tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5652be64-c2a3-4b24-a3aa-c962f204c06e] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:45.721475 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c97722f9-5ceb-4b9f-869f-9cb116d7732b tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:45.722091 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c97722f9-5ceb-4b9f-869f-9cb116d7732b tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:45.722576 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c97722f9-5ceb-4b9f-869f-9cb116d7732b tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:45.723334 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c97722f9-5ceb-4b9f-869f-9cb116d7732b tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:45.723799 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c97722f9-5ceb-4b9f-869f-9cb116d7732b tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:45.724214 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c97722f9-5ceb-4b9f-869f-9cb116d7732b tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:45.810707 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.objects.instance [None req-c97722f9-5ceb-4b9f-869f-9cb116d7732b tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lazy-loading 'services' on Instance uuid 5652be64-c2a3-4b24-a3aa-c962f204c06e {{(pid=118962) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 30 01:03:45.896372 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-c97722f9-5ceb-4b9f-869f-9cb116d7732b tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "DELETE /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e/os-volume_attachments/00c66df2-3c81-4adf-8028-8c47eb8b88ba" status: 202 len: 0 microversion: 2.1 time: 0.185840 Oct 30 01:03:45.896950 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 441/881] 10.222.0.16 () {58 vars in 1416 bytes} [Sun Oct 30 01:03:45 2022] DELETE /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e/os-volume_attachments/00c66df2-3c81-4adf-8028-8c47eb8b88ba => generated 0 bytes in 187 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 30 01:03:45.905794 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-7ba070f7-829a-4282-8530-122e95579871 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:45.908041 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-7ba070f7-829a-4282-8530-122e95579871 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5652be64-c2a3-4b24-a3aa-c962f204c06e] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:45.915660 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7ba070f7-829a-4282-8530-122e95579871 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:45.916273 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7ba070f7-829a-4282-8530-122e95579871 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:45.916890 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7ba070f7-829a-4282-8530-122e95579871 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:45.917605 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7ba070f7-829a-4282-8530-122e95579871 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:45.918116 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7ba070f7-829a-4282-8530-122e95579871 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:45.918739 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7ba070f7-829a-4282-8530-122e95579871 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:45.949314 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-7ba070f7-829a-4282-8530-122e95579871 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047535 Oct 30 01:03:45.949886 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 441/882] 10.222.0.16 () {58 vars in 1302 bytes} [Sun Oct 30 01:03:45 2022] GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:03:46.496356 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-7cc58b22-be73-4455-8bed-27ca4b143cf5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "5eb9e9ce-78a3-49b0-b099-d73664815017"}} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:03:46.500370 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-7cc58b22-be73-4455-8bed-27ca4b143cf5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:46.507829 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7cc58b22-be73-4455-8bed-27ca4b143cf5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:46.508309 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7cc58b22-be73-4455-8bed-27ca4b143cf5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:46.508737 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7cc58b22-be73-4455-8bed-27ca4b143cf5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:46.509426 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7cc58b22-be73-4455-8bed-27ca4b143cf5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:46.509872 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7cc58b22-be73-4455-8bed-27ca4b143cf5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:46.510263 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7cc58b22-be73-4455-8bed-27ca4b143cf5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:46.886901 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-7cc58b22-be73-4455-8bed-27ca4b143cf5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "POST /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.566083 Oct 30 01:03:46.887500 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 442/883] 10.222.0.16 () {60 vars in 1323 bytes} [Sun Oct 30 01:03:46 2022] POST /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de/os-volume_attachments => generated 194 bytes in 568 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:03:46.962039 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-1dd5aad8-ef90-4044-9a51-37f2391241db tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:46.964202 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-1dd5aad8-ef90-4044-9a51-37f2391241db tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5652be64-c2a3-4b24-a3aa-c962f204c06e] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:46.970802 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1dd5aad8-ef90-4044-9a51-37f2391241db tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:46.971278 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1dd5aad8-ef90-4044-9a51-37f2391241db tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:46.971703 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1dd5aad8-ef90-4044-9a51-37f2391241db tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:46.972500 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1dd5aad8-ef90-4044-9a51-37f2391241db tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:46.972953 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1dd5aad8-ef90-4044-9a51-37f2391241db tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:46.973366 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1dd5aad8-ef90-4044-9a51-37f2391241db tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:47.009565 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-1dd5aad8-ef90-4044-9a51-37f2391241db tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051803 Oct 30 01:03:47.010319 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 442/884] 10.222.0.16 () {58 vars in 1302 bytes} [Sun Oct 30 01:03:46 2022] GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:03:48.027013 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-6eda4131-0626-41cb-9a9d-c9cd4b07fc78 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:48.029263 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-6eda4131-0626-41cb-9a9d-c9cd4b07fc78 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5652be64-c2a3-4b24-a3aa-c962f204c06e] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:48.035537 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6eda4131-0626-41cb-9a9d-c9cd4b07fc78 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:48.035957 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6eda4131-0626-41cb-9a9d-c9cd4b07fc78 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:48.036379 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6eda4131-0626-41cb-9a9d-c9cd4b07fc78 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:48.037230 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6eda4131-0626-41cb-9a9d-c9cd4b07fc78 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:48.037673 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6eda4131-0626-41cb-9a9d-c9cd4b07fc78 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:48.038082 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6eda4131-0626-41cb-9a9d-c9cd4b07fc78 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:48.066654 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-6eda4131-0626-41cb-9a9d-c9cd4b07fc78 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.044529 Oct 30 01:03:48.067205 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 443/885] 10.222.0.16 () {58 vars in 1302 bytes} [Sun Oct 30 01:03:48 2022] GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e/os-volume_attachments => generated 25 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 30 01:03:48.135110 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-9e4de920-779e-42fb-829b-c5ec29ed5f67 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:48.135637 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-9e4de920-779e-42fb-829b-c5ec29ed5f67 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5652be64-c2a3-4b24-a3aa-c962f204c06e] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:48.142561 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9e4de920-779e-42fb-829b-c5ec29ed5f67 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:48.143040 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9e4de920-779e-42fb-829b-c5ec29ed5f67 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:48.143469 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9e4de920-779e-42fb-829b-c5ec29ed5f67 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:48.144192 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9e4de920-779e-42fb-829b-c5ec29ed5f67 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:48.144640 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9e4de920-779e-42fb-829b-c5ec29ed5f67 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:48.145059 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9e4de920-779e-42fb-829b-c5ec29ed5f67 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:48.171537 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-9e4de920-779e-42fb-829b-c5ec29ed5f67 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5652be64-c2a3-4b24-a3aa-c962f204c06e] Going to try to terminate instance {{(pid=118961) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 30 01:03:48.250182 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-9e4de920-779e-42fb-829b-c5ec29ed5f67 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "DELETE /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e" status: 204 len: 0 microversion: 2.1 time: 0.118765 Oct 30 01:03:48.250948 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 443/886] 10.222.0.16 () {58 vars in 1239 bytes} [Sun Oct 30 01:03:48 2022] DELETE /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e => generated 0 bytes in 121 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 30 01:03:48.259483 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-d8989329-e3b4-4e75-b466-05d070e56b72 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:48.260112 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-d8989329-e3b4-4e75-b466-05d070e56b72 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5652be64-c2a3-4b24-a3aa-c962f204c06e] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:48.266503 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d8989329-e3b4-4e75-b466-05d070e56b72 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:48.266971 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d8989329-e3b4-4e75-b466-05d070e56b72 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:48.267392 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d8989329-e3b4-4e75-b466-05d070e56b72 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:48.268104 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d8989329-e3b4-4e75-b466-05d070e56b72 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:48.268544 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d8989329-e3b4-4e75-b466-05d070e56b72 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:48.268955 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d8989329-e3b4-4e75-b466-05d070e56b72 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:48.407608 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-d8989329-e3b4-4e75-b466-05d070e56b72 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:48.419363 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-d8989329-e3b4-4e75-b466-05d070e56b72 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e" status: 200 len: 1681 microversion: 2.1 time: 0.163481 Oct 30 01:03:48.420202 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 444/887] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:48 2022] GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e => generated 1681 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:49.443751 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-5dfcffc6-c455-464a-8562-caeeb1b70fc9 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:49.444627 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-5dfcffc6-c455-464a-8562-caeeb1b70fc9 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5652be64-c2a3-4b24-a3aa-c962f204c06e] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:49.451861 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5dfcffc6-c455-464a-8562-caeeb1b70fc9 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:49.452342 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5dfcffc6-c455-464a-8562-caeeb1b70fc9 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:49.452773 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5dfcffc6-c455-464a-8562-caeeb1b70fc9 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:49.453455 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5dfcffc6-c455-464a-8562-caeeb1b70fc9 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:49.453912 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5dfcffc6-c455-464a-8562-caeeb1b70fc9 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:49.454328 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5dfcffc6-c455-464a-8562-caeeb1b70fc9 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:49.586001 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-5dfcffc6-c455-464a-8562-caeeb1b70fc9 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:49.594026 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-5dfcffc6-c455-464a-8562-caeeb1b70fc9 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e" status: 200 len: 1681 microversion: 2.1 time: 0.154742 Oct 30 01:03:49.594658 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 444/888] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:49 2022] GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e => generated 1681 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:50.263030 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [req-d20239ef-a191-4574-b669-0017ef233f09 req-29b87a9a-5cad-4543-95e7-7667e380c7b8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5652be64-c2a3-4b24-a3aa-c962f204c06e", "name": "network-vif-deleted", "tag": "9218be5d-9ff8-4e15-b073-e87bc5e0401c"}]} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:03:50.273968 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-d20239ef-a191-4574-b669-0017ef233f09 req-29b87a9a-5cad-4543-95e7-7667e380c7b8 service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:50.274547 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-d20239ef-a191-4574-b669-0017ef233f09 req-29b87a9a-5cad-4543-95e7-7667e380c7b8 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:50.275249 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-d20239ef-a191-4574-b669-0017ef233f09 req-29b87a9a-5cad-4543-95e7-7667e380c7b8 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:50.286773 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.compute.server_external_events [req-d20239ef-a191-4574-b669-0017ef233f09 req-29b87a9a-5cad-4543-95e7-7667e380c7b8 service nova] Creating event network-vif-deleted:9218be5d-9ff8-4e15-b073-e87bc5e0401c for instance 5652be64-c2a3-4b24-a3aa-c962f204c06e on hv-cinder-maste Oct 30 01:03:50.292098 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [req-d20239ef-a191-4574-b669-0017ef233f09 req-29b87a9a-5cad-4543-95e7-7667e380c7b8 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032853 Oct 30 01:03:50.292936 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 445/889] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:03:50 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:03:50.617178 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-8aad3ff6-c037-472a-a577-6f6ef9b33470 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:50.618373 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-8aad3ff6-c037-472a-a577-6f6ef9b33470 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5652be64-c2a3-4b24-a3aa-c962f204c06e] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:50.627832 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8aad3ff6-c037-472a-a577-6f6ef9b33470 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:50.628319 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8aad3ff6-c037-472a-a577-6f6ef9b33470 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:50.628954 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8aad3ff6-c037-472a-a577-6f6ef9b33470 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:50.629676 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8aad3ff6-c037-472a-a577-6f6ef9b33470 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:50.630133 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8aad3ff6-c037-472a-a577-6f6ef9b33470 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:50.630956 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8aad3ff6-c037-472a-a577-6f6ef9b33470 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:50.664685 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8aad3ff6-c037-472a-a577-6f6ef9b33470 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:50.665177 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8aad3ff6-c037-472a-a577-6f6ef9b33470 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:50.665616 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8aad3ff6-c037-472a-a577-6f6ef9b33470 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:50.666070 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.objects.instance [None req-8aad3ff6-c037-472a-a577-6f6ef9b33470 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lazy-loading 'fault' on Instance uuid 5652be64-c2a3-4b24-a3aa-c962f204c06e {{(pid=118961) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 30 01:03:50.710548 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-8aad3ff6-c037-472a-a577-6f6ef9b33470 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:50.718652 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-8aad3ff6-c037-472a-a577-6f6ef9b33470 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e" status: 200 len: 1332 microversion: 2.1 time: 0.105628 Oct 30 01:03:50.719170 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 445/890] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:50 2022] GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e => generated 1332 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:51.748448 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-da6ba2a4-a1bb-4325-ac2a-6a7778f7b9ed tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:51.749449 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-da6ba2a4-a1bb-4325-ac2a-6a7778f7b9ed tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5652be64-c2a3-4b24-a3aa-c962f204c06e] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:51.759421 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-da6ba2a4-a1bb-4325-ac2a-6a7778f7b9ed tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:51.759925 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-da6ba2a4-a1bb-4325-ac2a-6a7778f7b9ed tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:51.760352 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-da6ba2a4-a1bb-4325-ac2a-6a7778f7b9ed tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:51.761034 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-da6ba2a4-a1bb-4325-ac2a-6a7778f7b9ed tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:51.761469 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-da6ba2a4-a1bb-4325-ac2a-6a7778f7b9ed tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:51.761887 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-da6ba2a4-a1bb-4325-ac2a-6a7778f7b9ed tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:51.775380 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.wsgi [None req-da6ba2a4-a1bb-4325-ac2a-6a7778f7b9ed tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] HTTP exception thrown: Instance 5652be64-c2a3-4b24-a3aa-c962f204c06e could not be found. Oct 30 01:03:51.777051 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-da6ba2a4-a1bb-4325-ac2a-6a7778f7b9ed tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Request method failure captured: Oct 30 01:03:51.777051 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: request: GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e HTTP/1.1 Oct 30 01:03:51.777051 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept: application/json Oct 30 01:03:51.777051 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept-Encoding: identity Oct 30 01:03:51.777051 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Connection: close Oct 30 01:03:51.777051 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Length: 0 Oct 30 01:03:51.777051 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Type: application/json Oct 30 01:03:51.777051 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Host: 10.222.0.16 Oct 30 01:03:51.777051 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Openstack-System-Scope: None Oct 30 01:03:51.777051 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: User-Agent: python-urllib3/1.26.12 Oct 30 01:03:51.777051 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Auth-Token: gAAAAABjXcuedsaRjtnPRcHo4HqTnifOYe1vcHHjSzl7t8SsEKfOt41JBob3t9Ien606ucEZmcgmpf5H0lzlL6oZS4vLpZERLur9y2YRhqd9AYPyhbrmUkrYmKAssv6PSCysf6z3-myn1YKr62Nxqfs7t1lZGZnZuH_I2j8LHl5PnVXWJxyADCY Oct 30 01:03:51.777051 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Id: None Oct 30 01:03:51.777051 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Name: None Oct 30 01:03:51.777051 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Identity-Status: Confirmed Oct 30 01:03:51.777051 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Is-Admin-Project: True Oct 30 01:03:51.777051 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Id: default Oct 30 01:03:51.777051 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Name: Default Oct 30 01:03:51.777051 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Id: 6b68b886c19c45d585409651f33b8626 Oct 30 01:03:51.777051 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Name: tempest-AttachVolumeNegativeTest-618009204 Oct 30 01:03:51.777051 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Role: reader,member Oct 30 01:03:51.777051 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Roles: reader,member Oct 30 01:03:51.778572 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_6b68b886c19c45d585409651f33b8626"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6b68b886c19c45d585409651f33b8626"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/6b68b886c19c45d585409651f33b8626"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6b68b886c19c45d585409651f33b8626"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 01:03:51.778572 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant: tempest-AttachVolumeNegativeTest-618009204 Oct 30 01:03:51.778572 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Id: 6b68b886c19c45d585409651f33b8626 Oct 30 01:03:51.778572 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-618009204 Oct 30 01:03:51.778572 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User: tempest-AttachVolumeNegativeTest-618009204-project Oct 30 01:03:51.778572 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Id: default Oct 30 01:03:51.778572 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Name: Default Oct 30 01:03:51.778572 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Id: ecaf8ec105c04b919256f852da859eba Oct 30 01:03:51.778572 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Name: tempest-AttachVolumeNegativeTest-618009204-project Oct 30 01:03:51.778572 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: method: > Oct 30 01:03:51.778572 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: exception: Instance 5652be64-c2a3-4b24-a3aa-c962f204c06e could not be found. Oct 30 01:03:51.778572 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: action_args: {'id': '5652be64-c2a3-4b24-a3aa-c962f204c06e'} Oct 30 01:03:51.779562 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 01:03:51.779562 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:03:51.779562 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 30 01:03:51.779562 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 30 01:03:51.779562 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 30 01:03:51.779562 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 30 01:03:51.779562 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 30 01:03:51.779562 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 30 01:03:51.779562 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 30 01:03:51.779562 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 30 01:03:51.779562 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5652be64-c2a3-4b24-a3aa-c962f204c06e could not be found. Oct 30 01:03:51.779562 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:03:51.779562 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:03:51.779562 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:03:51.779562 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:03:51.779562 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:03:51.779562 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:03:51.779562 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 01:03:51.779562 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 01:03:51.779562 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 01:03:51.781082 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 01:03:51.781082 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 30 01:03:51.781082 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 30 01:03:51.781082 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 30 01:03:51.781082 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 30 01:03:51.781082 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 30 01:03:51.781082 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 30 01:03:51.781082 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5652be64-c2a3-4b24-a3aa-c962f204c06e could not be found. Oct 30 01:03:51.781082 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:03:51.781082 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:03:51.781082 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:03:51.781082 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:03:51.781082 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:03:51.781082 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:03:51.781082 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 01:03:51.781082 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 01:03:51.781082 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5652be64-c2a3-4b24-a3aa-c962f204c06e could not be found. Oct 30 01:03:51.781082 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:03:51.782454 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-da6ba2a4-a1bb-4325-ac2a-6a7778f7b9ed tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Returning 404 to user: Instance 5652be64-c2a3-4b24-a3aa-c962f204c06e could not be found. {{(pid=118962) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 01:03:51.782454 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-da6ba2a4-a1bb-4325-ac2a-6a7778f7b9ed tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e" status: 404 len: 111 microversion: 2.1 time: 0.037993 Oct 30 01:03:51.782454 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 446/891] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:51 2022] GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e => generated 111 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 30 01:03:51.787623 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-5bc39ced-dea4-40b1-9019-6b6c4f022514 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '.version_select at 0x7f0eab994160>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:51.810624 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-5bc39ced-dea4-40b1-9019-6b6c4f022514 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1396989907" status: 202 len: 0 microversion: 2.1 time: 0.027098 Oct 30 01:03:51.811194 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 446/892] 10.222.0.16 () {58 vars in 1221 bytes} [Sun Oct 30 01:03:51 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1396989907 => generated 0 bytes in 28 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 30 01:03:52.428753 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-6738f99d-9e14-43f9-bacf-6c59011ecdf6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:52.429235 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-6738f99d-9e14-43f9-bacf-6c59011ecdf6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:52.435971 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6738f99d-9e14-43f9-bacf-6c59011ecdf6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:52.436450 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6738f99d-9e14-43f9-bacf-6c59011ecdf6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:52.436857 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6738f99d-9e14-43f9-bacf-6c59011ecdf6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:52.437586 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6738f99d-9e14-43f9-bacf-6c59011ecdf6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:52.438039 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6738f99d-9e14-43f9-bacf-6c59011ecdf6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:52.438455 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6738f99d-9e14-43f9-bacf-6c59011ecdf6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:52.470306 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-0bf8d004-4086-4926-b16f-42b58ce33ebe tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Action: 'create', calling method: .version_select at 0x7f0eab9c54c0>, body: {"keypair": {"name": "tempest-keypair-165130656", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBIPGSBZS1OfEgOW2d65NdYKvFJeJYoifMopoILrAJO52XxPdY3PReaqTT2rKUjBK5x2rtKfU86AtVbVFBMjKgBP4Ejgbk2CHES3k626X8cNaGVgXyjve7UcRhR7M98s9tQ=="}} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:03:52.470690 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-6738f99d-9e14-43f9-bacf-6c59011ecdf6 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f/os-volume_attachments/44b5cbd4-240a-46bd-bc05-26ae29cce184" status: 200 len: 194 microversion: 2.20 time: 0.046238 Oct 30 01:03:52.471586 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 447/893] 10.222.0.16 () {60 vars in 1454 bytes} [Sun Oct 30 01:03:52 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f/os-volume_attachments/44b5cbd4-240a-46bd-bc05-26ae29cce184 => generated 194 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 30 01:03:52.487240 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.quota [None req-0bf8d004-4086-4926-b16f-42b58ce33ebe tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Getting quotas for project 6b68b886c19c45d585409651f33b8626. Resources: {'key_pairs'} {{(pid=118961) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 30 01:03:52.491157 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.quota [None req-0bf8d004-4086-4926-b16f-42b58ce33ebe tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Getting quotas for user ecaf8ec105c04b919256f852da859eba and project 6b68b886c19c45d585409651f33b8626. Resources: {'key_pairs'} {{(pid=118961) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 30 01:03:52.514267 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-0bf8d004-4086-4926-b16f-42b58ce33ebe tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.047844 Oct 30 01:03:52.514872 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 447/894] 10.222.0.16 () {60 vars in 1159 bytes} [Sun Oct 30 01:03:52 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:03:52.542070 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-a0459644-9a32-4cf6-98b6-a498801d1730 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:52.542571 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-a0459644-9a32-4cf6-98b6-a498801d1730 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:52.548632 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a0459644-9a32-4cf6-98b6-a498801d1730 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:52.549106 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a0459644-9a32-4cf6-98b6-a498801d1730 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:52.549533 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a0459644-9a32-4cf6-98b6-a498801d1730 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:52.550220 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a0459644-9a32-4cf6-98b6-a498801d1730 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:52.550664 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a0459644-9a32-4cf6-98b6-a498801d1730 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:52.551079 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a0459644-9a32-4cf6-98b6-a498801d1730 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:52.635246 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.objects.instance [None req-a0459644-9a32-4cf6-98b6-a498801d1730 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lazy-loading 'services' on Instance uuid 9491a8f4-2b76-4a4e-a6bd-6720114e482f {{(pid=118962) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 30 01:03:52.723149 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-a0459644-9a32-4cf6-98b6-a498801d1730 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "DELETE /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f/os-volume_attachments/44b5cbd4-240a-46bd-bc05-26ae29cce184" status: 202 len: 0 microversion: 2.20 time: 0.184559 Oct 30 01:03:52.723877 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 448/895] 10.222.0.16 () {60 vars in 1457 bytes} [Sun Oct 30 01:03:52 2022] DELETE /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f/os-volume_attachments/44b5cbd4-240a-46bd-bc05-26ae29cce184 => generated 0 bytes in 187 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 30 01:03:52.734232 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-178a5858-badf-4ad7-90a8-f3a69f744971 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:52.736583 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-178a5858-badf-4ad7-90a8-f3a69f744971 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:52.744180 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-178a5858-badf-4ad7-90a8-f3a69f744971 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:52.744917 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-178a5858-badf-4ad7-90a8-f3a69f744971 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:52.745662 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-178a5858-badf-4ad7-90a8-f3a69f744971 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:52.747296 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-178a5858-badf-4ad7-90a8-f3a69f744971 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:52.748004 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-178a5858-badf-4ad7-90a8-f3a69f744971 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:52.749214 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-178a5858-badf-4ad7-90a8-f3a69f744971 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:52.783866 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-178a5858-badf-4ad7-90a8-f3a69f744971 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.054156 Oct 30 01:03:52.784451 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 448/896] 10.222.0.16 () {60 vars in 1343 bytes} [Sun Oct 30 01:03:52 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 30 01:03:53.798493 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-1e55430a-ae4f-4c06-b123-1eeedce64592 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:53.800895 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-1e55430a-ae4f-4c06-b123-1eeedce64592 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:53.807276 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1e55430a-ae4f-4c06-b123-1eeedce64592 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:53.807837 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1e55430a-ae4f-4c06-b123-1eeedce64592 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:53.808263 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1e55430a-ae4f-4c06-b123-1eeedce64592 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:53.808951 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1e55430a-ae4f-4c06-b123-1eeedce64592 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:53.809410 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1e55430a-ae4f-4c06-b123-1eeedce64592 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:53.809806 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1e55430a-ae4f-4c06-b123-1eeedce64592 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:53.840750 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-1e55430a-ae4f-4c06-b123-1eeedce64592 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047238 Oct 30 01:03:53.841324 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 449/897] 10.222.0.16 () {60 vars in 1343 bytes} [Sun Oct 30 01:03:53 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 30 01:03:53.890509 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-e90a8e6e-fc9c-4920-9999-cddb745615b2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1801127058", "imageRef": "4c16f4f6-84cb-4463-b82c-b7c65f3826f0", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "4400f15d-ace0-4893-b5ff-c43f47dad5a5"}], "security_groups": [{"name": "tempest-securitygroup--712391800"}], "key_name": "tempest-keypair-165130656", "user_data": "}} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:03:53.899586 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_api:SELECT=62,nova_api:DELETE=2,nova_api:UPDATE=1,nova_api:INSERT=1 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:03:54.005662 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.network.neutron [None req-e90a8e6e-fc9c-4920-9999-cddb745615b2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] validate_networks() for [('4400f15d-ace0-4893-b5ff-c43f47dad5a5', None, None, None, None, None)] {{(pid=118961) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 30 01:03:54.390161 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-e90a8e6e-fc9c-4920-9999-cddb745615b2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Flavor limits 0:0:0 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 30 01:03:54.390611 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-e90a8e6e-fc9c-4920-9999-cddb745615b2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Image limits 0:0:0 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 30 01:03:54.391189 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-e90a8e6e-fc9c-4920-9999-cddb745615b2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Flavor pref 0:0:0 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 30 01:03:54.391676 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-e90a8e6e-fc9c-4920-9999-cddb745615b2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Image pref 0:0:0 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 30 01:03:54.392179 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-e90a8e6e-fc9c-4920-9999-cddb745615b2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 30 01:03:54.401416 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e90a8e6e-fc9c-4920-9999-cddb745615b2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:54.401891 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e90a8e6e-fc9c-4920-9999-cddb745615b2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:54.402315 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e90a8e6e-fc9c-4920-9999-cddb745615b2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:54.403017 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e90a8e6e-fc9c-4920-9999-cddb745615b2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:54.403461 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e90a8e6e-fc9c-4920-9999-cddb745615b2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:54.403905 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e90a8e6e-fc9c-4920-9999-cddb745615b2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:54.420306 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.quota [None req-e90a8e6e-fc9c-4920-9999-cddb745615b2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Getting quotas for project 6b68b886c19c45d585409651f33b8626. Resources: {'ram', 'instances', 'cores'} {{(pid=118961) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 30 01:03:54.424883 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.quota [None req-e90a8e6e-fc9c-4920-9999-cddb745615b2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Getting quotas for user ecaf8ec105c04b919256f852da859eba and project 6b68b886c19c45d585409651f33b8626. Resources: {'ram', 'instances', 'cores'} {{(pid=118961) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 30 01:03:54.436772 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-e90a8e6e-fc9c-4920-9999-cddb745615b2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Going to run 1 instances... {{(pid=118961) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 30 01:03:54.439147 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-e90a8e6e-fc9c-4920-9999-cddb745615b2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: b77a1841-4f87-4b8a-96a9-a619ea67d452] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4c16f4f6-84cb-4463-b82c-b7c65f3826f0',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118961) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 30 01:03:54.480866 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-c1e20289-5c55-4ec2-bb65-73d988ba21d8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:03:54.483133 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-c1e20289-5c55-4ec2-bb65-73d988ba21d8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:54.490024 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c1e20289-5c55-4ec2-bb65-73d988ba21d8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:54.490533 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c1e20289-5c55-4ec2-bb65-73d988ba21d8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:54.491026 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-e90a8e6e-fc9c-4920-9999-cddb745615b2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.604936 Oct 30 01:03:54.491584 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c1e20289-5c55-4ec2-bb65-73d988ba21d8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:54.498168 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 449/898] 10.222.0.16 () {60 vars in 1147 bytes} [Sun Oct 30 01:03:53 2022] POST /compute/v2.1/servers => generated 401 bytes in 607 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 30 01:03:54.498326 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c1e20289-5c55-4ec2-bb65-73d988ba21d8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:54.498326 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c1e20289-5c55-4ec2-bb65-73d988ba21d8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:54.498326 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c1e20289-5c55-4ec2-bb65-73d988ba21d8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:54.504139 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-6defca96-8a45-41ab-b00c-abd2524a0b50 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:54.504755 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-6defca96-8a45-41ab-b00c-abd2524a0b50 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: b77a1841-4f87-4b8a-96a9-a619ea67d452] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:54.522239 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.objects.instance [None req-c1e20289-5c55-4ec2-bb65-73d988ba21d8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lazy-loading 'flavor' on Instance uuid de41579d-0173-4dba-962d-a28c85c505de {{(pid=118962) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 30 01:03:54.567030 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-6defca96-8a45-41ab-b00c-abd2524a0b50 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:54.574488 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-6defca96-8a45-41ab-b00c-abd2524a0b50 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452" status: 200 len: 1241 microversion: 2.1 time: 0.074398 Oct 30 01:03:54.575261 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 450/899] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:54 2022] GET /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452 => generated 1241 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:54.614177 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell1:SELECT=92,nova_cell1:UPDATE=4,nova_cell1:INSERT=4 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:03:54.669757 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-c1e20289-5c55-4ec2-bb65-73d988ba21d8 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "POST /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de/action" status: 200 len: 32 microversion: 2.1 time: 0.192938 Oct 30 01:03:54.670601 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 450/900] 10.222.0.16 () {60 vars in 1278 bytes} [Sun Oct 30 01:03:54 2022] POST /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de/action => generated 32 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Oct 30 01:03:54.680617 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-4c246dfd-a0a0-4042-954a-888e4f0c1929 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:54.681249 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-4c246dfd-a0a0-4042-954a-888e4f0c1929 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:54.689023 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4c246dfd-a0a0-4042-954a-888e4f0c1929 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:54.689535 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4c246dfd-a0a0-4042-954a-888e4f0c1929 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:54.689977 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4c246dfd-a0a0-4042-954a-888e4f0c1929 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:54.690873 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4c246dfd-a0a0-4042-954a-888e4f0c1929 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:54.691328 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4c246dfd-a0a0-4042-954a-888e4f0c1929 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:54.691771 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4c246dfd-a0a0-4042-954a-888e4f0c1929 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:54.854791 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-f27895b3-98c8-4666-a7d2-e5828cfb582e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:54.859054 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-f27895b3-98c8-4666-a7d2-e5828cfb582e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:54.877563 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f27895b3-98c8-4666-a7d2-e5828cfb582e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:54.878710 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f27895b3-98c8-4666-a7d2-e5828cfb582e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:54.879404 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f27895b3-98c8-4666-a7d2-e5828cfb582e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:54.880542 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f27895b3-98c8-4666-a7d2-e5828cfb582e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:54.881491 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f27895b3-98c8-4666-a7d2-e5828cfb582e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:54.882083 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f27895b3-98c8-4666-a7d2-e5828cfb582e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:54.921785 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-f27895b3-98c8-4666-a7d2-e5828cfb582e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.071622 Oct 30 01:03:54.922633 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 451/901] 10.222.0.16 () {60 vars in 1343 bytes} [Sun Oct 30 01:03:54 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f/os-volume_attachments => generated 197 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 30 01:03:55.021918 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-4c246dfd-a0a0-4042-954a-888e4f0c1929 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:55.034033 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-4c246dfd-a0a0-4042-954a-888e4f0c1929 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1736 microversion: 2.1 time: 0.357708 Oct 30 01:03:55.041052 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 451/902] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:54 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1736 bytes in 365 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:55.600196 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-9c135cec-3336-4f96-95fd-3927105fad60 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:55.600491 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-9c135cec-3336-4f96-95fd-3927105fad60 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: b77a1841-4f87-4b8a-96a9-a619ea67d452] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:55.603887 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_api:SELECT=51,nova_api:UPDATE=1 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:03:55.607587 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9c135cec-3336-4f96-95fd-3927105fad60 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:55.608084 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9c135cec-3336-4f96-95fd-3927105fad60 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:55.608510 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9c135cec-3336-4f96-95fd-3927105fad60 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:55.609197 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9c135cec-3336-4f96-95fd-3927105fad60 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:55.609637 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9c135cec-3336-4f96-95fd-3927105fad60 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:55.610046 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9c135cec-3336-4f96-95fd-3927105fad60 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:55.681357 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-9c135cec-3336-4f96-95fd-3927105fad60 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:55.693114 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-9c135cec-3336-4f96-95fd-3927105fad60 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452" status: 200 len: 1293 microversion: 2.1 time: 0.098098 Oct 30 01:03:55.693867 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 452/903] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:55 2022] GET /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452 => generated 1293 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:55.937859 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-6cffd14a-c126-42eb-a9c0-675da86fe49e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:55.941083 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-6cffd14a-c126-42eb-a9c0-675da86fe49e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:55.951600 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6cffd14a-c126-42eb-a9c0-675da86fe49e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:55.952390 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6cffd14a-c126-42eb-a9c0-675da86fe49e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:55.953069 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6cffd14a-c126-42eb-a9c0-675da86fe49e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:55.954095 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6cffd14a-c126-42eb-a9c0-675da86fe49e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:55.954780 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6cffd14a-c126-42eb-a9c0-675da86fe49e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:55.955416 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6cffd14a-c126-42eb-a9c0-675da86fe49e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:55.986582 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-6cffd14a-c126-42eb-a9c0-675da86fe49e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.055104 Oct 30 01:03:55.987117 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 452/904] 10.222.0.16 () {60 vars in 1343 bytes} [Sun Oct 30 01:03:55 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f/os-volume_attachments => generated 25 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:56.060665 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-ed1ecedc-f448-48db-886a-7ca7af776301 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:56.061505 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-ed1ecedc-f448-48db-886a-7ca7af776301 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:56.068541 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ed1ecedc-f448-48db-886a-7ca7af776301 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:56.069347 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ed1ecedc-f448-48db-886a-7ca7af776301 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:56.070035 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ed1ecedc-f448-48db-886a-7ca7af776301 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:56.070793 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ed1ecedc-f448-48db-886a-7ca7af776301 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:56.071230 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ed1ecedc-f448-48db-886a-7ca7af776301 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:56.071468 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-381471e7-0736-4248-93fc-16254b67ab59 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:56.072075 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-381471e7-0736-4248-93fc-16254b67ab59 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:56.072587 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ed1ecedc-f448-48db-886a-7ca7af776301 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:56.078576 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-381471e7-0736-4248-93fc-16254b67ab59 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:56.079218 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-381471e7-0736-4248-93fc-16254b67ab59 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:56.079957 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-381471e7-0736-4248-93fc-16254b67ab59 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:56.080796 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-381471e7-0736-4248-93fc-16254b67ab59 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:56.081348 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-381471e7-0736-4248-93fc-16254b67ab59 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:56.081871 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-381471e7-0736-4248-93fc-16254b67ab59 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:56.110484 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-381471e7-0736-4248-93fc-16254b67ab59 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Going to try to terminate instance {{(pid=118961) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 30 01:03:56.211283 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-381471e7-0736-4248-93fc-16254b67ab59 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "DELETE /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 204 len: 0 microversion: 2.20 time: 0.143561 Oct 30 01:03:56.214931 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 453/905] 10.222.0.16 () {60 vars in 1280 bytes} [Sun Oct 30 01:03:56 2022] DELETE /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 0 bytes in 154 msecs (HTTP/1.1 204) 9 headers in 365 bytes (2 switches on core 0) Oct 30 01:03:56.224772 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-4bd27a41-0686-4e73-8495-e00afa413c58 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:56.225527 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-4bd27a41-0686-4e73-8495-e00afa413c58 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:56.231204 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-ed1ecedc-f448-48db-886a-7ca7af776301 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:56.231895 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4bd27a41-0686-4e73-8495-e00afa413c58 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:56.232382 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4bd27a41-0686-4e73-8495-e00afa413c58 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:56.232870 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4bd27a41-0686-4e73-8495-e00afa413c58 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:56.233787 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4bd27a41-0686-4e73-8495-e00afa413c58 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:56.234245 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4bd27a41-0686-4e73-8495-e00afa413c58 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:56.234717 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4bd27a41-0686-4e73-8495-e00afa413c58 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:56.242798 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-ed1ecedc-f448-48db-886a-7ca7af776301 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1736 microversion: 2.1 time: 0.186440 Oct 30 01:03:56.243387 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 453/906] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:56 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1736 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:56.375847 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-4bd27a41-0686-4e73-8495-e00afa413c58 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:56.384139 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-4bd27a41-0686-4e73-8495-e00afa413c58 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:56.385558 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-4bd27a41-0686-4e73-8495-e00afa413c58 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:56.386722 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-4bd27a41-0686-4e73-8495-e00afa413c58 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1723 microversion: 2.20 time: 0.166432 Oct 30 01:03:56.387242 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 454/907] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:03:56 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1723 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 30 01:03:56.721593 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-f1f44f26-4685-4dd6-a03d-92913eb628b9 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:56.722595 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-f1f44f26-4685-4dd6-a03d-92913eb628b9 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: b77a1841-4f87-4b8a-96a9-a619ea67d452] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:56.732865 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f1f44f26-4685-4dd6-a03d-92913eb628b9 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:56.733555 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f1f44f26-4685-4dd6-a03d-92913eb628b9 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:56.734258 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f1f44f26-4685-4dd6-a03d-92913eb628b9 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:56.735384 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f1f44f26-4685-4dd6-a03d-92913eb628b9 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:56.736175 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f1f44f26-4685-4dd6-a03d-92913eb628b9 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:56.736866 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f1f44f26-4685-4dd6-a03d-92913eb628b9 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:56.802847 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-f1f44f26-4685-4dd6-a03d-92913eb628b9 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:56.810715 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-f1f44f26-4685-4dd6-a03d-92913eb628b9 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452" status: 200 len: 1299 microversion: 2.1 time: 0.095721 Oct 30 01:03:56.811289 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 454/908] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:56 2022] GET /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452 => generated 1299 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:57.270653 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-e00f7e0b-9ba5-4e1f-805d-0f476a1066e1 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:57.271690 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-e00f7e0b-9ba5-4e1f-805d-0f476a1066e1 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:57.279324 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e00f7e0b-9ba5-4e1f-805d-0f476a1066e1 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:57.279848 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e00f7e0b-9ba5-4e1f-805d-0f476a1066e1 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:57.280280 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e00f7e0b-9ba5-4e1f-805d-0f476a1066e1 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:57.280995 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e00f7e0b-9ba5-4e1f-805d-0f476a1066e1 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:57.281443 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e00f7e0b-9ba5-4e1f-805d-0f476a1066e1 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:57.282007 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e00f7e0b-9ba5-4e1f-805d-0f476a1066e1 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:57.286465 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:03:57.415220 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-a3b1c43f-d2f9-4ddd-837f-c3eec328cc23 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:57.415970 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-a3b1c43f-d2f9-4ddd-837f-c3eec328cc23 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:57.422876 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a3b1c43f-d2f9-4ddd-837f-c3eec328cc23 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:57.423362 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a3b1c43f-d2f9-4ddd-837f-c3eec328cc23 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:57.423827 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a3b1c43f-d2f9-4ddd-837f-c3eec328cc23 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:57.424036 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-e00f7e0b-9ba5-4e1f-805d-0f476a1066e1 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:57.424528 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a3b1c43f-d2f9-4ddd-837f-c3eec328cc23 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:57.424974 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a3b1c43f-d2f9-4ddd-837f-c3eec328cc23 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:57.425392 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a3b1c43f-d2f9-4ddd-837f-c3eec328cc23 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:57.432751 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-e00f7e0b-9ba5-4e1f-805d-0f476a1066e1 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1736 microversion: 2.1 time: 0.168726 Oct 30 01:03:57.433342 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 455/909] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:57 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1736 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:57.572725 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-a3b1c43f-d2f9-4ddd-837f-c3eec328cc23 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:57.586227 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-a3b1c43f-d2f9-4ddd-837f-c3eec328cc23 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:57.587648 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-a3b1c43f-d2f9-4ddd-837f-c3eec328cc23 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:57.588835 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-a3b1c43f-d2f9-4ddd-837f-c3eec328cc23 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1723 microversion: 2.20 time: 0.177815 Oct 30 01:03:57.589372 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 455/910] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:03:57 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1723 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:03:57.844507 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-02f60f8b-62bc-430f-8a84-e814f369d467 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:57.847739 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-02f60f8b-62bc-430f-8a84-e814f369d467 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: b77a1841-4f87-4b8a-96a9-a619ea67d452] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:57.862940 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-02f60f8b-62bc-430f-8a84-e814f369d467 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:57.863437 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-02f60f8b-62bc-430f-8a84-e814f369d467 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:57.863904 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-02f60f8b-62bc-430f-8a84-e814f369d467 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:57.864617 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-02f60f8b-62bc-430f-8a84-e814f369d467 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:57.865071 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-02f60f8b-62bc-430f-8a84-e814f369d467 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:57.865501 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-02f60f8b-62bc-430f-8a84-e814f369d467 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:57.928339 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-02f60f8b-62bc-430f-8a84-e814f369d467 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:57.936479 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-02f60f8b-62bc-430f-8a84-e814f369d467 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452" status: 200 len: 1299 microversion: 2.1 time: 0.096735 Oct 30 01:03:57.937015 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 456/911] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:57 2022] GET /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452 => generated 1299 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:58.456520 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-def956d5-1759-463a-9a9f-d01f0f994484 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:58.457133 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-def956d5-1759-463a-9a9f-d01f0f994484 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:58.464044 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-def956d5-1759-463a-9a9f-d01f0f994484 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:58.464523 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-def956d5-1759-463a-9a9f-d01f0f994484 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:58.465137 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-def956d5-1759-463a-9a9f-d01f0f994484 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:58.465819 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-def956d5-1759-463a-9a9f-d01f0f994484 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:58.466441 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-def956d5-1759-463a-9a9f-d01f0f994484 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:58.466855 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-def956d5-1759-463a-9a9f-d01f0f994484 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:58.608929 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-def956d5-1759-463a-9a9f-d01f0f994484 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:58.614762 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-ae92559d-b14b-4e67-b947-b94b83d6032e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:58.615539 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-ae92559d-b14b-4e67-b947-b94b83d6032e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:58.619666 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-def956d5-1759-463a-9a9f-d01f0f994484 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1736 microversion: 2.1 time: 0.166838 Oct 30 01:03:58.626288 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ae92559d-b14b-4e67-b947-b94b83d6032e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:58.626800 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ae92559d-b14b-4e67-b947-b94b83d6032e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:58.627238 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ae92559d-b14b-4e67-b947-b94b83d6032e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:58.627988 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ae92559d-b14b-4e67-b947-b94b83d6032e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:58.628446 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ae92559d-b14b-4e67-b947-b94b83d6032e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:58.628866 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ae92559d-b14b-4e67-b947-b94b83d6032e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:58.632391 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 456/912] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:58 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1736 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:58.782284 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-ae92559d-b14b-4e67-b947-b94b83d6032e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:58.794680 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-ae92559d-b14b-4e67-b947-b94b83d6032e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:58.796945 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-ae92559d-b14b-4e67-b947-b94b83d6032e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:58.798665 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-ae92559d-b14b-4e67-b947-b94b83d6032e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1723 microversion: 2.20 time: 0.188089 Oct 30 01:03:58.799457 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 457/913] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:03:58 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1723 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:03:58.963209 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-5f3dee60-78ce-4a1e-9d6b-3d1a51dea9e6 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:58.963825 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-5f3dee60-78ce-4a1e-9d6b-3d1a51dea9e6 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: b77a1841-4f87-4b8a-96a9-a619ea67d452] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:58.970242 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5f3dee60-78ce-4a1e-9d6b-3d1a51dea9e6 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:58.970714 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5f3dee60-78ce-4a1e-9d6b-3d1a51dea9e6 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:58.971165 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5f3dee60-78ce-4a1e-9d6b-3d1a51dea9e6 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:58.971873 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5f3dee60-78ce-4a1e-9d6b-3d1a51dea9e6 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:58.972319 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5f3dee60-78ce-4a1e-9d6b-3d1a51dea9e6 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:58.972740 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5f3dee60-78ce-4a1e-9d6b-3d1a51dea9e6 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:59.104183 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-5f3dee60-78ce-4a1e-9d6b-3d1a51dea9e6 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:59.112927 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-5f3dee60-78ce-4a1e-9d6b-3d1a51dea9e6 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452" status: 200 len: 1366 microversion: 2.1 time: 0.154345 Oct 30 01:03:59.113668 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 457/914] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:58 2022] GET /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452 => generated 1366 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:59.645341 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-965e2b27-cc53-49f8-922c-e7e39b095676 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:59.645920 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-965e2b27-cc53-49f8-922c-e7e39b095676 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:59.653162 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-965e2b27-cc53-49f8-922c-e7e39b095676 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:59.653656 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-965e2b27-cc53-49f8-922c-e7e39b095676 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:59.654181 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-965e2b27-cc53-49f8-922c-e7e39b095676 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:59.654930 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-965e2b27-cc53-49f8-922c-e7e39b095676 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:59.655387 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-965e2b27-cc53-49f8-922c-e7e39b095676 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:59.655841 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-965e2b27-cc53-49f8-922c-e7e39b095676 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:59.792832 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-965e2b27-cc53-49f8-922c-e7e39b095676 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:59.801264 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-965e2b27-cc53-49f8-922c-e7e39b095676 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1736 microversion: 2.1 time: 0.160159 Oct 30 01:03:59.802053 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 458/915] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:03:59 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1736 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:03:59.826281 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-65993159-2c79-4b92-98d5-3bdae5a0882c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:03:59.826862 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-65993159-2c79-4b92-98d5-3bdae5a0882c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:03:59.833338 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-65993159-2c79-4b92-98d5-3bdae5a0882c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:59.834136 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-65993159-2c79-4b92-98d5-3bdae5a0882c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:59.834792 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-65993159-2c79-4b92-98d5-3bdae5a0882c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:59.835488 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-65993159-2c79-4b92-98d5-3bdae5a0882c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:03:59.836003 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-65993159-2c79-4b92-98d5-3bdae5a0882c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:03:59.836619 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-65993159-2c79-4b92-98d5-3bdae5a0882c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:03:59.969548 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-65993159-2c79-4b92-98d5-3bdae5a0882c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:59.978577 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-65993159-2c79-4b92-98d5-3bdae5a0882c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:59.980038 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-65993159-2c79-4b92-98d5-3bdae5a0882c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:03:59.981220 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-65993159-2c79-4b92-98d5-3bdae5a0882c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1723 microversion: 2.20 time: 0.158917 Oct 30 01:03:59.981783 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 458/916] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:03:59 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1723 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:04:00.136045 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-052635f7-47c6-4994-be90-0fedf3d94722 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:00.136976 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-052635f7-47c6-4994-be90-0fedf3d94722 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: b77a1841-4f87-4b8a-96a9-a619ea67d452] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:00.143339 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-052635f7-47c6-4994-be90-0fedf3d94722 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:00.144135 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-052635f7-47c6-4994-be90-0fedf3d94722 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:00.144776 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-052635f7-47c6-4994-be90-0fedf3d94722 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:00.145833 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-052635f7-47c6-4994-be90-0fedf3d94722 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:00.146492 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-052635f7-47c6-4994-be90-0fedf3d94722 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:00.147107 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-052635f7-47c6-4994-be90-0fedf3d94722 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:00.287084 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [req-7a739736-c293-41f3-9bc7-8e45d4bb5fdd req-a4015784-1437-4297-988a-2cf357130477 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b77a1841-4f87-4b8a-96a9-a619ea67d452", "tag": "a89f7199-cd3e-44d3-af70-4235cf4ab1a1"}]} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:04:00.290929 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-052635f7-47c6-4994-be90-0fedf3d94722 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:00.299024 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-052635f7-47c6-4994-be90-0fedf3d94722 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452" status: 200 len: 1366 microversion: 2.1 time: 0.167041 Oct 30 01:04:00.302450 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 459/917] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:04:00 2022] GET /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452 => generated 1366 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:04:00.322513 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-7a739736-c293-41f3-9bc7-8e45d4bb5fdd req-a4015784-1437-4297-988a-2cf357130477 service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:00.322981 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-7a739736-c293-41f3-9bc7-8e45d4bb5fdd req-a4015784-1437-4297-988a-2cf357130477 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:00.323396 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-7a739736-c293-41f3-9bc7-8e45d4bb5fdd req-a4015784-1437-4297-988a-2cf357130477 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:00.334651 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.compute.server_external_events [req-7a739736-c293-41f3-9bc7-8e45d4bb5fdd req-a4015784-1437-4297-988a-2cf357130477 service nova] Creating event network-changed:a89f7199-cd3e-44d3-af70-4235cf4ab1a1 for instance b77a1841-4f87-4b8a-96a9-a619ea67d452 on hv-cinder-maste Oct 30 01:04:00.340240 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [req-7a739736-c293-41f3-9bc7-8e45d4bb5fdd req-a4015784-1437-4297-988a-2cf357130477 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.058456 Oct 30 01:04:00.350529 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 459/918] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:04:00 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:04:00.825166 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-9ecf7edf-a297-4d4e-a060-1e99d0997580 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:00.826308 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-9ecf7edf-a297-4d4e-a060-1e99d0997580 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:00.833457 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9ecf7edf-a297-4d4e-a060-1e99d0997580 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:00.834312 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9ecf7edf-a297-4d4e-a060-1e99d0997580 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:00.834743 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9ecf7edf-a297-4d4e-a060-1e99d0997580 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:00.835668 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9ecf7edf-a297-4d4e-a060-1e99d0997580 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:00.836319 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9ecf7edf-a297-4d4e-a060-1e99d0997580 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:00.836926 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9ecf7edf-a297-4d4e-a060-1e99d0997580 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:00.987875 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-9ecf7edf-a297-4d4e-a060-1e99d0997580 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:00.996477 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-9ecf7edf-a297-4d4e-a060-1e99d0997580 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1736 microversion: 2.1 time: 0.175172 Oct 30 01:04:00.997048 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 460/919] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:04:00 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1736 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:04:01.007910 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-74ebc71e-9c4b-4eea-ad7e-b457f8b8ddc1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:01.008682 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-74ebc71e-9c4b-4eea-ad7e-b457f8b8ddc1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:01.015271 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-74ebc71e-9c4b-4eea-ad7e-b457f8b8ddc1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:01.015778 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-74ebc71e-9c4b-4eea-ad7e-b457f8b8ddc1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:01.016209 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-74ebc71e-9c4b-4eea-ad7e-b457f8b8ddc1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:01.016900 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-74ebc71e-9c4b-4eea-ad7e-b457f8b8ddc1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:01.017347 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-74ebc71e-9c4b-4eea-ad7e-b457f8b8ddc1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:01.017789 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-74ebc71e-9c4b-4eea-ad7e-b457f8b8ddc1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:01.049754 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-74ebc71e-9c4b-4eea-ad7e-b457f8b8ddc1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:01.050130 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-74ebc71e-9c4b-4eea-ad7e-b457f8b8ddc1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:01.050560 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-74ebc71e-9c4b-4eea-ad7e-b457f8b8ddc1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:01.051004 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.objects.instance [None req-74ebc71e-9c4b-4eea-ad7e-b457f8b8ddc1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lazy-loading 'fault' on Instance uuid 9491a8f4-2b76-4a4e-a6bd-6720114e482f {{(pid=118962) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 30 01:04:01.095084 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-74ebc71e-9c4b-4eea-ad7e-b457f8b8ddc1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:01.103397 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-74ebc71e-9c4b-4eea-ad7e-b457f8b8ddc1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:01.105122 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-74ebc71e-9c4b-4eea-ad7e-b457f8b8ddc1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:01.106305 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-74ebc71e-9c4b-4eea-ad7e-b457f8b8ddc1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1372 microversion: 2.20 time: 0.103477 Oct 30 01:04:01.119740 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 460/920] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:04:01 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1372 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:04:01.322901 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-272689e6-0412-4782-97a9-c33ccfdf2d19 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:01.323756 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-272689e6-0412-4782-97a9-c33ccfdf2d19 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: b77a1841-4f87-4b8a-96a9-a619ea67d452] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:01.331922 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-272689e6-0412-4782-97a9-c33ccfdf2d19 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:01.332811 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-272689e6-0412-4782-97a9-c33ccfdf2d19 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:01.333250 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-272689e6-0412-4782-97a9-c33ccfdf2d19 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:01.334346 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-272689e6-0412-4782-97a9-c33ccfdf2d19 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:01.334783 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-272689e6-0412-4782-97a9-c33ccfdf2d19 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:01.335197 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-272689e6-0412-4782-97a9-c33ccfdf2d19 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:01.473360 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-272689e6-0412-4782-97a9-c33ccfdf2d19 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:01.481658 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-272689e6-0412-4782-97a9-c33ccfdf2d19 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452" status: 200 len: 1366 microversion: 2.1 time: 0.163108 Oct 30 01:04:01.482201 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 461/921] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:04:01 2022] GET /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452 => generated 1366 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:04:02.024368 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-a358285d-d6de-47f3-ad03-048cfbe32f31 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:02.025062 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-a358285d-d6de-47f3-ad03-048cfbe32f31 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:02.032633 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a358285d-d6de-47f3-ad03-048cfbe32f31 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:02.033171 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a358285d-d6de-47f3-ad03-048cfbe32f31 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:02.033649 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a358285d-d6de-47f3-ad03-048cfbe32f31 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:02.034409 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a358285d-d6de-47f3-ad03-048cfbe32f31 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:02.034901 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a358285d-d6de-47f3-ad03-048cfbe32f31 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:02.035358 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a358285d-d6de-47f3-ad03-048cfbe32f31 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:02.132547 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-ca659a80-2ddb-42e9-8a06-f959287fd2d3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:02.133682 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-ca659a80-2ddb-42e9-8a06-f959287fd2d3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:02.139729 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ca659a80-2ddb-42e9-8a06-f959287fd2d3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:02.140233 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ca659a80-2ddb-42e9-8a06-f959287fd2d3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:02.140673 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ca659a80-2ddb-42e9-8a06-f959287fd2d3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:02.141366 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ca659a80-2ddb-42e9-8a06-f959287fd2d3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:02.141812 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ca659a80-2ddb-42e9-8a06-f959287fd2d3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:02.142223 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ca659a80-2ddb-42e9-8a06-f959287fd2d3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:02.166429 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-a358285d-d6de-47f3-ad03-048cfbe32f31 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:02.174454 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ca659a80-2ddb-42e9-8a06-f959287fd2d3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:02.174931 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ca659a80-2ddb-42e9-8a06-f959287fd2d3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:02.175359 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ca659a80-2ddb-42e9-8a06-f959287fd2d3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:02.175818 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.objects.instance [None req-ca659a80-2ddb-42e9-8a06-f959287fd2d3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lazy-loading 'fault' on Instance uuid 9491a8f4-2b76-4a4e-a6bd-6720114e482f {{(pid=118961) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 30 01:04:02.176281 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-a358285d-d6de-47f3-ad03-048cfbe32f31 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1736 microversion: 2.1 time: 0.158190 Oct 30 01:04:02.176931 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 461/922] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:04:02 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1736 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:04:02.216959 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-ca659a80-2ddb-42e9-8a06-f959287fd2d3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:02.225152 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-ca659a80-2ddb-42e9-8a06-f959287fd2d3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:02.226654 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-ca659a80-2ddb-42e9-8a06-f959287fd2d3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:02.227840 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-ca659a80-2ddb-42e9-8a06-f959287fd2d3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1372 microversion: 2.20 time: 0.099314 Oct 30 01:04:02.228409 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 462/923] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:04:02 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1372 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 30 01:04:02.369787 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [req-b0646259-e881-4067-928b-8f0a1f8cfc95 req-1d9c7eef-b429-4dab-951a-002d614d2a43 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9491a8f4-2b76-4a4e-a6bd-6720114e482f", "name": "network-vif-deleted", "tag": "5367850e-73d0-4814-af96-bd430c94c272"}]} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:04:02.380051 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-b0646259-e881-4067-928b-8f0a1f8cfc95 req-1d9c7eef-b429-4dab-951a-002d614d2a43 service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:02.380517 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-b0646259-e881-4067-928b-8f0a1f8cfc95 req-1d9c7eef-b429-4dab-951a-002d614d2a43 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:02.380932 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-b0646259-e881-4067-928b-8f0a1f8cfc95 req-1d9c7eef-b429-4dab-951a-002d614d2a43 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:02.392556 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.compute.server_external_events [req-b0646259-e881-4067-928b-8f0a1f8cfc95 req-1d9c7eef-b429-4dab-951a-002d614d2a43 service nova] Creating event network-vif-deleted:5367850e-73d0-4814-af96-bd430c94c272 for instance 9491a8f4-2b76-4a4e-a6bd-6720114e482f on hv-cinder-maste Oct 30 01:04:02.397392 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [req-b0646259-e881-4067-928b-8f0a1f8cfc95 req-1d9c7eef-b429-4dab-951a-002d614d2a43 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031890 Oct 30 01:04:02.397992 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 462/924] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:04:02 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:04:02.508694 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-ec447256-f0f5-49d3-af7d-ec24703b3f15 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:02.509706 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-ec447256-f0f5-49d3-af7d-ec24703b3f15 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: b77a1841-4f87-4b8a-96a9-a619ea67d452] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:02.519922 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ec447256-f0f5-49d3-af7d-ec24703b3f15 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:02.520729 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ec447256-f0f5-49d3-af7d-ec24703b3f15 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:02.521456 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ec447256-f0f5-49d3-af7d-ec24703b3f15 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:02.522603 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ec447256-f0f5-49d3-af7d-ec24703b3f15 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:02.523349 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ec447256-f0f5-49d3-af7d-ec24703b3f15 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:02.524120 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ec447256-f0f5-49d3-af7d-ec24703b3f15 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:02.656470 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-ec447256-f0f5-49d3-af7d-ec24703b3f15 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:02.669120 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-ec447256-f0f5-49d3-af7d-ec24703b3f15 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452" status: 200 len: 1366 microversion: 2.1 time: 0.166652 Oct 30 01:04:02.669919 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 463/925] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:04:02 2022] GET /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452 => generated 1366 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:04:03.202200 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-207bc2ea-b7a6-4f7c-8264-872fd3460ae5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:03.202973 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-207bc2ea-b7a6-4f7c-8264-872fd3460ae5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:03.211107 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-207bc2ea-b7a6-4f7c-8264-872fd3460ae5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:03.211675 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-207bc2ea-b7a6-4f7c-8264-872fd3460ae5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:03.212225 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-207bc2ea-b7a6-4f7c-8264-872fd3460ae5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:03.212992 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-207bc2ea-b7a6-4f7c-8264-872fd3460ae5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:03.213490 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-207bc2ea-b7a6-4f7c-8264-872fd3460ae5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:03.213954 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-207bc2ea-b7a6-4f7c-8264-872fd3460ae5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:03.256178 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-77979a5c-b8b2-46bb-8901-f0f0ad504d9c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:03.256884 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-77979a5c-b8b2-46bb-8901-f0f0ad504d9c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:03.266096 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-77979a5c-b8b2-46bb-8901-f0f0ad504d9c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:03.266601 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-77979a5c-b8b2-46bb-8901-f0f0ad504d9c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:03.267642 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-77979a5c-b8b2-46bb-8901-f0f0ad504d9c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:03.268702 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-77979a5c-b8b2-46bb-8901-f0f0ad504d9c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:03.269301 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-77979a5c-b8b2-46bb-8901-f0f0ad504d9c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:03.269826 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-77979a5c-b8b2-46bb-8901-f0f0ad504d9c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:03.303637 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-77979a5c-b8b2-46bb-8901-f0f0ad504d9c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:03.304157 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-77979a5c-b8b2-46bb-8901-f0f0ad504d9c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:03.304591 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-77979a5c-b8b2-46bb-8901-f0f0ad504d9c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:03.305038 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.objects.instance [None req-77979a5c-b8b2-46bb-8901-f0f0ad504d9c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lazy-loading 'fault' on Instance uuid 9491a8f4-2b76-4a4e-a6bd-6720114e482f {{(pid=118961) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 30 01:04:03.356153 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-77979a5c-b8b2-46bb-8901-f0f0ad504d9c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:03.370598 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-77979a5c-b8b2-46bb-8901-f0f0ad504d9c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:03.373006 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-77979a5c-b8b2-46bb-8901-f0f0ad504d9c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:03.374261 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-77979a5c-b8b2-46bb-8901-f0f0ad504d9c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1372 microversion: 2.20 time: 0.124257 Oct 30 01:04:03.379828 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 464/926] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:04:03 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1372 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:04:03.380152 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-207bc2ea-b7a6-4f7c-8264-872fd3460ae5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:03.386271 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-207bc2ea-b7a6-4f7c-8264-872fd3460ae5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1736 microversion: 2.1 time: 0.188797 Oct 30 01:04:03.387322 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 463/927] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:04:03 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1736 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:04:03.699016 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-bf690fdd-e266-4cfe-96f1-fe632cc90d86 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:03.699855 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-bf690fdd-e266-4cfe-96f1-fe632cc90d86 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: b77a1841-4f87-4b8a-96a9-a619ea67d452] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:03.708152 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bf690fdd-e266-4cfe-96f1-fe632cc90d86 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:03.708789 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bf690fdd-e266-4cfe-96f1-fe632cc90d86 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:03.709721 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bf690fdd-e266-4cfe-96f1-fe632cc90d86 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:03.710661 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bf690fdd-e266-4cfe-96f1-fe632cc90d86 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:03.711277 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bf690fdd-e266-4cfe-96f1-fe632cc90d86 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:03.711987 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bf690fdd-e266-4cfe-96f1-fe632cc90d86 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:03.850718 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-bf690fdd-e266-4cfe-96f1-fe632cc90d86 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:03.859089 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-bf690fdd-e266-4cfe-96f1-fe632cc90d86 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452" status: 200 len: 1366 microversion: 2.1 time: 0.165612 Oct 30 01:04:03.859754 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 465/928] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:04:03 2022] GET /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452 => generated 1366 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:04:04.400661 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-64d75ec0-7426-4ae8-9b2b-5f68764bbbc8 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:04.401237 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-64d75ec0-7426-4ae8-9b2b-5f68764bbbc8 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:04.408252 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-64d75ec0-7426-4ae8-9b2b-5f68764bbbc8 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:04.408763 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-64d75ec0-7426-4ae8-9b2b-5f68764bbbc8 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:04.409175 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-9f407474-3f1e-42f0-9cac-8a56e392923d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:04.409628 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-64d75ec0-7426-4ae8-9b2b-5f68764bbbc8 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:04.409786 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-9f407474-3f1e-42f0-9cac-8a56e392923d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:04.409956 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-64d75ec0-7426-4ae8-9b2b-5f68764bbbc8 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:04.410311 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-64d75ec0-7426-4ae8-9b2b-5f68764bbbc8 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:04.410730 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-64d75ec0-7426-4ae8-9b2b-5f68764bbbc8 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:04.412184 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell0:SELECT=2 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:04:04.418217 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9f407474-3f1e-42f0-9cac-8a56e392923d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:04.418700 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9f407474-3f1e-42f0-9cac-8a56e392923d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:04.419125 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9f407474-3f1e-42f0-9cac-8a56e392923d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:04.419844 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9f407474-3f1e-42f0-9cac-8a56e392923d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:04.420293 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9f407474-3f1e-42f0-9cac-8a56e392923d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:04.420710 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9f407474-3f1e-42f0-9cac-8a56e392923d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:04.444211 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-64d75ec0-7426-4ae8-9b2b-5f68764bbbc8 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:04.445663 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-64d75ec0-7426-4ae8-9b2b-5f68764bbbc8 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:04.446449 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-64d75ec0-7426-4ae8-9b2b-5f68764bbbc8 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:04.446977 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.objects.instance [None req-64d75ec0-7426-4ae8-9b2b-5f68764bbbc8 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lazy-loading 'fault' on Instance uuid 9491a8f4-2b76-4a4e-a6bd-6720114e482f {{(pid=118962) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 30 01:04:04.513983 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-64d75ec0-7426-4ae8-9b2b-5f68764bbbc8 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:04.523608 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-64d75ec0-7426-4ae8-9b2b-5f68764bbbc8 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:04.525921 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-64d75ec0-7426-4ae8-9b2b-5f68764bbbc8 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:04.533621 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-64d75ec0-7426-4ae8-9b2b-5f68764bbbc8 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 200 len: 1372 microversion: 2.20 time: 0.137028 Oct 30 01:04:04.534712 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 464/929] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:04:04 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 1372 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:04:04.574057 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell0:SELECT=12 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:04:04.585627 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-9f407474-3f1e-42f0-9cac-8a56e392923d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:04.595480 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-9f407474-3f1e-42f0-9cac-8a56e392923d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1736 microversion: 2.1 time: 0.190117 Oct 30 01:04:04.599485 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 466/930] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:04:04 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1736 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:04:04.882594 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-1dd6f1ac-0c69-4e44-ab74-ff9bc7bc8d0c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:04.883423 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-1dd6f1ac-0c69-4e44-ab74-ff9bc7bc8d0c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: b77a1841-4f87-4b8a-96a9-a619ea67d452] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:04.890342 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1dd6f1ac-0c69-4e44-ab74-ff9bc7bc8d0c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:04.891053 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1dd6f1ac-0c69-4e44-ab74-ff9bc7bc8d0c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:04.891481 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1dd6f1ac-0c69-4e44-ab74-ff9bc7bc8d0c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:04.892220 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1dd6f1ac-0c69-4e44-ab74-ff9bc7bc8d0c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:04.892654 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1dd6f1ac-0c69-4e44-ab74-ff9bc7bc8d0c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:04.893118 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1dd6f1ac-0c69-4e44-ab74-ff9bc7bc8d0c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:05.034347 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-1dd6f1ac-0c69-4e44-ab74-ff9bc7bc8d0c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:05.042375 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-1dd6f1ac-0c69-4e44-ab74-ff9bc7bc8d0c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452" status: 200 len: 1366 microversion: 2.1 time: 0.163918 Oct 30 01:04:05.042962 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 465/931] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:04:04 2022] GET /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452 => generated 1366 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:04:05.520003 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [req-bd5cf075-636b-4949-bdc0-18527345ce5c req-f211b409-6dd4-43b8-97e4-abb4419644be service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b77a1841-4f87-4b8a-96a9-a619ea67d452", "name": "network-vif-plugged", "status": "completed", "tag": "a89f7199-cd3e-44d3-af70-4235cf4ab1a1"}]} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:04:05.532069 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-bd5cf075-636b-4949-bdc0-18527345ce5c req-f211b409-6dd4-43b8-97e4-abb4419644be service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:05.532669 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-bd5cf075-636b-4949-bdc0-18527345ce5c req-f211b409-6dd4-43b8-97e4-abb4419644be service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:05.533160 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-bd5cf075-636b-4949-bdc0-18527345ce5c req-f211b409-6dd4-43b8-97e4-abb4419644be service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:05.545755 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.compute.server_external_events [req-bd5cf075-636b-4949-bdc0-18527345ce5c req-f211b409-6dd4-43b8-97e4-abb4419644be service nova] Creating event network-vif-plugged:a89f7199-cd3e-44d3-af70-4235cf4ab1a1 for instance b77a1841-4f87-4b8a-96a9-a619ea67d452 on hv-cinder-maste Oct 30 01:04:05.551161 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [req-bd5cf075-636b-4949-bdc0-18527345ce5c req-f211b409-6dd4-43b8-97e4-abb4419644be service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036992 Oct 30 01:04:05.552355 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 467/932] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:04:05 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:04:05.562880 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-7f901ad5-7d34-487f-a192-ce2b3b9492b3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:05.563489 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-7f901ad5-7d34-487f-a192-ce2b3b9492b3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:05.569687 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7f901ad5-7d34-487f-a192-ce2b3b9492b3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:05.570164 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7f901ad5-7d34-487f-a192-ce2b3b9492b3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:05.570595 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7f901ad5-7d34-487f-a192-ce2b3b9492b3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:05.571286 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7f901ad5-7d34-487f-a192-ce2b3b9492b3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:05.571751 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7f901ad5-7d34-487f-a192-ce2b3b9492b3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:05.572319 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7f901ad5-7d34-487f-a192-ce2b3b9492b3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:05.586467 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.wsgi [None req-7f901ad5-7d34-487f-a192-ce2b3b9492b3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] HTTP exception thrown: Instance 9491a8f4-2b76-4a4e-a6bd-6720114e482f could not be found. Oct 30 01:04:05.588263 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-7f901ad5-7d34-487f-a192-ce2b3b9492b3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Request method failure captured: Oct 30 01:04:05.588263 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: request: GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f HTTP/1.1 Oct 30 01:04:05.588263 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept: application/json Oct 30 01:04:05.588263 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept-Encoding: identity Oct 30 01:04:05.588263 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Connection: close Oct 30 01:04:05.588263 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Length: 0 Oct 30 01:04:05.588263 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Type: application/json Oct 30 01:04:05.588263 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Host: 10.222.0.16 Oct 30 01:04:05.588263 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Openstack-System-Scope: None Oct 30 01:04:05.588263 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: User-Agent: python-urllib3/1.26.12 Oct 30 01:04:05.588263 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Auth-Token: gAAAAABjXcugVWKyquPgiWxxmLIR7tjn5wgo5PMehRN68UdChNzQYP0dt_zzBfBJguIoFG_dA8xfdVXJfHfs12PVvciy_RYdtFnj0iZgJFLLfVRiSCDdkIwZN-XNhM2aC5XCKhmUA8zgKICHzu0rW4fXBpKKyrnf0oul0CZJ7cPc4A8OWNhd6Jg Oct 30 01:04:05.588263 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Id: None Oct 30 01:04:05.588263 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Name: None Oct 30 01:04:05.588263 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Identity-Status: Confirmed Oct 30 01:04:05.588263 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Is-Admin-Project: True Oct 30 01:04:05.588263 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Openstack-Nova-Api-Version: 2.20 Oct 30 01:04:05.588263 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Id: default Oct 30 01:04:05.588263 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Name: Default Oct 30 01:04:05.588263 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Id: a3bfc7112bb04760aa4e9adc50b12361 Oct 30 01:04:05.588263 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-40368566 Oct 30 01:04:05.588263 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Role: reader,member Oct 30 01:04:05.588263 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Roles: reader,member Oct 30 01:04:05.589649 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_a3bfc7112bb04760aa4e9adc50b12361"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a3bfc7112bb04760aa4e9adc50b12361"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/a3bfc7112bb04760aa4e9adc50b12361"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a3bfc7112bb04760aa4e9adc50b12361"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 01:04:05.589649 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant: tempest-AttachVolumeShelveTestJSON-40368566 Oct 30 01:04:05.589649 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Id: a3bfc7112bb04760aa4e9adc50b12361 Oct 30 01:04:05.589649 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-40368566 Oct 30 01:04:05.589649 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User: tempest-AttachVolumeShelveTestJSON-40368566-project Oct 30 01:04:05.589649 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Id: default Oct 30 01:04:05.589649 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Name: Default Oct 30 01:04:05.589649 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Id: 15e603f9898a430d8a53f21818e97722 Oct 30 01:04:05.589649 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Name: tempest-AttachVolumeShelveTestJSON-40368566-project Oct 30 01:04:05.589649 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: method: > Oct 30 01:04:05.589649 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: exception: Instance 9491a8f4-2b76-4a4e-a6bd-6720114e482f could not be found. Oct 30 01:04:05.589649 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: action_args: {'id': '9491a8f4-2b76-4a4e-a6bd-6720114e482f'} Oct 30 01:04:05.590404 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 01:04:05.590404 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:04:05.590404 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 30 01:04:05.590404 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 30 01:04:05.590404 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 30 01:04:05.590404 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 30 01:04:05.590404 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 30 01:04:05.590404 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 30 01:04:05.590404 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 30 01:04:05.590404 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 30 01:04:05.590404 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9491a8f4-2b76-4a4e-a6bd-6720114e482f could not be found. Oct 30 01:04:05.590404 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:04:05.590404 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:04:05.590404 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:04:05.590404 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:04:05.590404 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:04:05.590404 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:04:05.590404 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 01:04:05.590404 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 01:04:05.590404 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 01:04:05.592587 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 01:04:05.592587 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 30 01:04:05.592587 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 30 01:04:05.592587 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 30 01:04:05.592587 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 30 01:04:05.592587 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 30 01:04:05.592587 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 30 01:04:05.592587 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9491a8f4-2b76-4a4e-a6bd-6720114e482f could not be found. Oct 30 01:04:05.592587 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:04:05.592587 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:04:05.592587 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:04:05.592587 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:04:05.592587 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:04:05.592587 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:04:05.592587 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 01:04:05.592587 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 01:04:05.592587 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9491a8f4-2b76-4a4e-a6bd-6720114e482f could not be found. Oct 30 01:04:05.592587 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:04:05.594913 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-7f901ad5-7d34-487f-a192-ce2b3b9492b3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Returning 404 to user: Instance 9491a8f4-2b76-4a4e-a6bd-6720114e482f could not be found. {{(pid=118962) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 01:04:05.594913 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-7f901ad5-7d34-487f-a192-ce2b3b9492b3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 404 len: 111 microversion: 2.20 time: 0.030860 Oct 30 01:04:05.594913 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 466/933] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:04:05 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 30 01:04:05.603924 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-64fb31b8-01c2-4ea5-a39e-86004f6c44eb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '.version_select at 0x7f0eaba33940>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:05.619277 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-38c59188-3bf9-4790-ab32-33304b812a5f tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:05.620456 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-38c59188-3bf9-4790-ab32-33304b812a5f tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:05.625166 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-64fb31b8-01c2-4ea5-a39e-86004f6c44eb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1576642984" status: 204 len: 0 microversion: 2.20 time: 0.028848 Oct 30 01:04:05.625834 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 468/934] 10.222.0.16 () {60 vars in 1262 bytes} [Sun Oct 30 01:04:05 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1576642984 => generated 0 bytes in 30 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 30 01:04:05.626652 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-38c59188-3bf9-4790-ab32-33304b812a5f tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:05.627153 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-38c59188-3bf9-4790-ab32-33304b812a5f tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:05.627550 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-38c59188-3bf9-4790-ab32-33304b812a5f tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:05.628274 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-38c59188-3bf9-4790-ab32-33304b812a5f tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:05.628719 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-38c59188-3bf9-4790-ab32-33304b812a5f tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:05.629131 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-38c59188-3bf9-4790-ab32-33304b812a5f tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:05.651442 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:04:05.781949 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-38c59188-3bf9-4790-ab32-33304b812a5f tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:05.790592 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-38c59188-3bf9-4790-ab32-33304b812a5f tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1736 microversion: 2.1 time: 0.175911 Oct 30 01:04:05.791504 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 467/935] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:04:05 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1736 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 01:04:06.065616 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-3aaa0075-31f6-48f6-86cf-13ee4da76242 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:06.066413 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-3aaa0075-31f6-48f6-86cf-13ee4da76242 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: b77a1841-4f87-4b8a-96a9-a619ea67d452] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:06.073079 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3aaa0075-31f6-48f6-86cf-13ee4da76242 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:06.073567 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3aaa0075-31f6-48f6-86cf-13ee4da76242 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:06.073980 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3aaa0075-31f6-48f6-86cf-13ee4da76242 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:06.074702 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3aaa0075-31f6-48f6-86cf-13ee4da76242 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:06.075150 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3aaa0075-31f6-48f6-86cf-13ee4da76242 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:06.075563 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3aaa0075-31f6-48f6-86cf-13ee4da76242 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:06.215740 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-3aaa0075-31f6-48f6-86cf-13ee4da76242 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:06.223750 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-3aaa0075-31f6-48f6-86cf-13ee4da76242 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452" status: 200 len: 1366 microversion: 2.1 time: 0.162006 Oct 30 01:04:06.224247 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 469/936] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:04:06 2022] GET /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452 => generated 1366 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:04:06.379526 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-8724de25-dcaa-432a-8d00-a45a14b9e7f2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Action: 'create', calling method: .version_select at 0x7f0eafb0ce50>, body: {"keypair": {"name": "tempest-keypair-872797247", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHqtNEbLhz/7b6MlRFrmbmApcmIAGtZ/2w6sRTpEPjpYcI0o0Un2nFwzUpnpB6tNQc8hqKjBazfJESlOVQd1AJ0P5g7vwUudvHcpHefrx0UbzWtmEtoF3we741CMUYMSxQ=="}} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:04:06.395205 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.quota [None req-8724de25-dcaa-432a-8d00-a45a14b9e7f2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Getting quotas for project a3bfc7112bb04760aa4e9adc50b12361. Resources: {'key_pairs'} {{(pid=118962) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 30 01:04:06.399460 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.quota [None req-8724de25-dcaa-432a-8d00-a45a14b9e7f2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Getting quotas for user 15e603f9898a430d8a53f21818e97722 and project a3bfc7112bb04760aa4e9adc50b12361. Resources: {'key_pairs'} {{(pid=118962) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 30 01:04:06.424876 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-8724de25-dcaa-432a-8d00-a45a14b9e7f2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 201 len: 400 microversion: 2.20 time: 0.049380 Oct 30 01:04:06.425512 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 468/937] 10.222.0.16 () {62 vars in 1200 bytes} [Sun Oct 30 01:04:06 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 51 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Oct 30 01:04:06.814589 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-0e128534-dbb5-4a24-84b0-3097742aa352 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:06.815210 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-0e128534-dbb5-4a24-84b0-3097742aa352 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:06.821982 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0e128534-dbb5-4a24-84b0-3097742aa352 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:06.822477 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0e128534-dbb5-4a24-84b0-3097742aa352 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:06.822912 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0e128534-dbb5-4a24-84b0-3097742aa352 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:06.823815 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0e128534-dbb5-4a24-84b0-3097742aa352 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:06.824305 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0e128534-dbb5-4a24-84b0-3097742aa352 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:06.824727 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0e128534-dbb5-4a24-84b0-3097742aa352 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:06.956837 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-0e128534-dbb5-4a24-84b0-3097742aa352 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:06.966181 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-0e128534-dbb5-4a24-84b0-3097742aa352 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1736 microversion: 2.1 time: 0.155407 Oct 30 01:04:06.966771 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 470/938] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:04:06 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1736 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:04:07.248292 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-ec061ae0-5cee-47d9-8df9-3a79d07d57be tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:07.248981 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-ec061ae0-5cee-47d9-8df9-3a79d07d57be tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: b77a1841-4f87-4b8a-96a9-a619ea67d452] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:07.255155 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ec061ae0-5cee-47d9-8df9-3a79d07d57be tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:07.255631 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ec061ae0-5cee-47d9-8df9-3a79d07d57be tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:07.256100 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ec061ae0-5cee-47d9-8df9-3a79d07d57be tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:07.256811 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ec061ae0-5cee-47d9-8df9-3a79d07d57be tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:07.257275 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ec061ae0-5cee-47d9-8df9-3a79d07d57be tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:07.257794 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ec061ae0-5cee-47d9-8df9-3a79d07d57be tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:07.634194 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-ec061ae0-5cee-47d9-8df9-3a79d07d57be tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:07.642264 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-ec061ae0-5cee-47d9-8df9-3a79d07d57be tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452" status: 200 len: 1366 microversion: 2.1 time: 0.399069 Oct 30 01:04:07.642921 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 469/939] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:04:07 2022] GET /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452 => generated 1366 bytes in 400 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:04:07.762534 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-2bac0332-f4a9-454b-b076-d1178c4992a4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-2100874067", "imageRef": "4c16f4f6-84cb-4463-b82c-b7c65f3826f0", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "8d646d29-e717-40c9-89d4-df0c802eb607"}], "security_groups": [{"name": "tempest-securitygroup--198697515"}], "key_name": "tempest-keypair-872797247", "user_data": "}} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:04:07.871464 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.network.neutron [None req-2bac0332-f4a9-454b-b076-d1178c4992a4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] validate_networks() for [('8d646d29-e717-40c9-89d4-df0c802eb607', None, None, None, None, None)] {{(pid=118961) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 30 01:04:07.990636 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-ea4f3034-85d4-4466-9c38-fe1816be33e4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:07.991265 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-ea4f3034-85d4-4466-9c38-fe1816be33e4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:07.997895 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ea4f3034-85d4-4466-9c38-fe1816be33e4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:08.002935 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ea4f3034-85d4-4466-9c38-fe1816be33e4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:08.003459 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ea4f3034-85d4-4466-9c38-fe1816be33e4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:08.005526 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ea4f3034-85d4-4466-9c38-fe1816be33e4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:08.005997 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ea4f3034-85d4-4466-9c38-fe1816be33e4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:08.006507 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ea4f3034-85d4-4466-9c38-fe1816be33e4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:08.222386 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-ea4f3034-85d4-4466-9c38-fe1816be33e4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:08.231209 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-ea4f3034-85d4-4466-9c38-fe1816be33e4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1736 microversion: 2.1 time: 0.244521 Oct 30 01:04:08.231792 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 470/940] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:04:07 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1736 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:04:08.325497 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-2bac0332-f4a9-454b-b076-d1178c4992a4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Flavor limits 0:0:0 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 30 01:04:08.325952 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-2bac0332-f4a9-454b-b076-d1178c4992a4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Image limits 0:0:0 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 30 01:04:08.326424 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-2bac0332-f4a9-454b-b076-d1178c4992a4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Flavor pref 0:0:0 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 30 01:04:08.326838 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-2bac0332-f4a9-454b-b076-d1178c4992a4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Image pref 0:0:0 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 30 01:04:08.327248 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-2bac0332-f4a9-454b-b076-d1178c4992a4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 30 01:04:08.336661 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2bac0332-f4a9-454b-b076-d1178c4992a4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:08.337137 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2bac0332-f4a9-454b-b076-d1178c4992a4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:08.337898 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2bac0332-f4a9-454b-b076-d1178c4992a4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:08.338667 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2bac0332-f4a9-454b-b076-d1178c4992a4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:08.339118 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2bac0332-f4a9-454b-b076-d1178c4992a4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:08.339537 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2bac0332-f4a9-454b-b076-d1178c4992a4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:08.357546 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.quota [None req-2bac0332-f4a9-454b-b076-d1178c4992a4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Getting quotas for project a3bfc7112bb04760aa4e9adc50b12361. Resources: {'ram', 'instances', 'cores'} {{(pid=118961) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 30 01:04:08.361749 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.quota [None req-2bac0332-f4a9-454b-b076-d1178c4992a4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Getting quotas for user 15e603f9898a430d8a53f21818e97722 and project a3bfc7112bb04760aa4e9adc50b12361. Resources: {'ram', 'instances', 'cores'} {{(pid=118961) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 30 01:04:08.372886 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-2bac0332-f4a9-454b-b076-d1178c4992a4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Going to run 1 instances... {{(pid=118961) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 30 01:04:08.375242 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-2bac0332-f4a9-454b-b076-d1178c4992a4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4c16f4f6-84cb-4463-b82c-b7c65f3826f0',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118961) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 30 01:04:08.425147 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-2bac0332-f4a9-454b-b076-d1178c4992a4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.20 time: 0.666822 Oct 30 01:04:08.425886 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 471/941] 10.222.0.16 () {62 vars in 1188 bytes} [Sun Oct 30 01:04:07 2022] POST /compute/v2.1/servers => generated 401 bytes in 668 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Oct 30 01:04:08.437003 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-e7969c41-9fb4-49b8-a3c4-b70e0e5f1e28 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:08.437605 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-e7969c41-9fb4-49b8-a3c4-b70e0e5f1e28 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:08.512106 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-e7969c41-9fb4-49b8-a3c4-b70e0e5f1e28 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:08.519883 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-e7969c41-9fb4-49b8-a3c4-b70e0e5f1e28 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:08.521308 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-e7969c41-9fb4-49b8-a3c4-b70e0e5f1e28 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:08.522417 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-e7969c41-9fb4-49b8-a3c4-b70e0e5f1e28 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1281 microversion: 2.20 time: 0.089368 Oct 30 01:04:08.522971 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 471/942] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:04:08 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1281 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:04:08.666342 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-b0a34342-14ad-4d4d-b662-1d8569f2f98e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:08.667069 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-b0a34342-14ad-4d4d-b662-1d8569f2f98e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: b77a1841-4f87-4b8a-96a9-a619ea67d452] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:08.674702 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b0a34342-14ad-4d4d-b662-1d8569f2f98e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:08.675531 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b0a34342-14ad-4d4d-b662-1d8569f2f98e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:08.676719 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b0a34342-14ad-4d4d-b662-1d8569f2f98e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:08.677641 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b0a34342-14ad-4d4d-b662-1d8569f2f98e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:08.678301 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b0a34342-14ad-4d4d-b662-1d8569f2f98e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:08.678968 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b0a34342-14ad-4d4d-b662-1d8569f2f98e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:08.819347 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-b0a34342-14ad-4d4d-b662-1d8569f2f98e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:08.829324 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-b0a34342-14ad-4d4d-b662-1d8569f2f98e tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452" status: 200 len: 1557 microversion: 2.1 time: 0.166970 Oct 30 01:04:08.829842 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 472/943] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:04:08 2022] GET /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452 => generated 1557 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:04:08.852324 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-a200c318-60f0-45f5-9e1e-d1870131aac0 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:08.852837 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-a200c318-60f0-45f5-9e1e-d1870131aac0 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: b77a1841-4f87-4b8a-96a9-a619ea67d452] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:08.858878 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a200c318-60f0-45f5-9e1e-d1870131aac0 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:08.859344 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a200c318-60f0-45f5-9e1e-d1870131aac0 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:08.859817 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a200c318-60f0-45f5-9e1e-d1870131aac0 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:08.860493 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a200c318-60f0-45f5-9e1e-d1870131aac0 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:08.860933 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a200c318-60f0-45f5-9e1e-d1870131aac0 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:08.861345 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a200c318-60f0-45f5-9e1e-d1870131aac0 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:08.939345 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-a200c318-60f0-45f5-9e1e-d1870131aac0 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.090965 Oct 30 01:04:08.939980 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 472/944] 10.222.0.16 () {58 vars in 1275 bytes} [Sun Oct 30 01:04:08 2022] GET /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452/os-interface => generated 285 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:04:09.255026 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-4b403c5c-3275-4d93-aaf5-156df4f78cb3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:09.256306 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-4b403c5c-3275-4d93-aaf5-156df4f78cb3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:09.262953 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4b403c5c-3275-4d93-aaf5-156df4f78cb3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:09.263482 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4b403c5c-3275-4d93-aaf5-156df4f78cb3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:09.264009 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4b403c5c-3275-4d93-aaf5-156df4f78cb3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:09.264732 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4b403c5c-3275-4d93-aaf5-156df4f78cb3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:09.265197 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4b403c5c-3275-4d93-aaf5-156df4f78cb3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:09.265625 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4b403c5c-3275-4d93-aaf5-156df4f78cb3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:09.284723 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell1:SELECT=100 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:04:09.512592 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-4b403c5c-3275-4d93-aaf5-156df4f78cb3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:09.521617 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-4b403c5c-3275-4d93-aaf5-156df4f78cb3 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1736 microversion: 2.1 time: 0.270368 Oct 30 01:04:09.522460 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 473/945] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:04:09 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1736 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:04:09.547633 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-9c7e0cde-a237-494c-83ab-ec63091fcf24 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:09.548345 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-9c7e0cde-a237-494c-83ab-ec63091fcf24 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:09.554506 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9c7e0cde-a237-494c-83ab-ec63091fcf24 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:09.554987 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9c7e0cde-a237-494c-83ab-ec63091fcf24 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:09.555416 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9c7e0cde-a237-494c-83ab-ec63091fcf24 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:09.556153 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9c7e0cde-a237-494c-83ab-ec63091fcf24 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:09.556647 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9c7e0cde-a237-494c-83ab-ec63091fcf24 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:09.557064 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9c7e0cde-a237-494c-83ab-ec63091fcf24 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:09.652996 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-9c7e0cde-a237-494c-83ab-ec63091fcf24 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:09.661598 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-9c7e0cde-a237-494c-83ab-ec63091fcf24 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:09.663033 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-9c7e0cde-a237-494c-83ab-ec63091fcf24 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:09.664182 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-9c7e0cde-a237-494c-83ab-ec63091fcf24 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1333 microversion: 2.20 time: 0.121098 Oct 30 01:04:09.664718 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 473/946] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:04:09 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1333 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:04:10.032551 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [req-8bec0d70-33bc-4f5c-b5d6-6010aa2799af req-8d4ca671-c02c-4ca6-9243-63153ec4473b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b77a1841-4f87-4b8a-96a9-a619ea67d452", "tag": "a89f7199-cd3e-44d3-af70-4235cf4ab1a1"}]} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:04:10.042440 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-8bec0d70-33bc-4f5c-b5d6-6010aa2799af req-8d4ca671-c02c-4ca6-9243-63153ec4473b service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:10.042942 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-8bec0d70-33bc-4f5c-b5d6-6010aa2799af req-8d4ca671-c02c-4ca6-9243-63153ec4473b service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:10.043372 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-8bec0d70-33bc-4f5c-b5d6-6010aa2799af req-8d4ca671-c02c-4ca6-9243-63153ec4473b service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:10.058510 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.compute.server_external_events [req-8bec0d70-33bc-4f5c-b5d6-6010aa2799af req-8d4ca671-c02c-4ca6-9243-63153ec4473b service nova] Creating event network-changed:a89f7199-cd3e-44d3-af70-4235cf4ab1a1 for instance b77a1841-4f87-4b8a-96a9-a619ea67d452 on hv-cinder-maste Oct 30 01:04:10.063796 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [req-8bec0d70-33bc-4f5c-b5d6-6010aa2799af req-8d4ca671-c02c-4ca6-9243-63153ec4473b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035592 Oct 30 01:04:10.064417 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 474/947] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:04:10 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 30 01:04:10.546448 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-2065e933-ffcd-4872-a305-3869ca3a4492 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:10.547486 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-2065e933-ffcd-4872-a305-3869ca3a4492 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:10.555653 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2065e933-ffcd-4872-a305-3869ca3a4492 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:10.556949 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2065e933-ffcd-4872-a305-3869ca3a4492 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:10.557555 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2065e933-ffcd-4872-a305-3869ca3a4492 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:10.558472 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2065e933-ffcd-4872-a305-3869ca3a4492 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:10.558947 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2065e933-ffcd-4872-a305-3869ca3a4492 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:10.559332 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2065e933-ffcd-4872-a305-3869ca3a4492 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:10.691019 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-d0f38d35-5647-4a2d-a72f-70e463a297a3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:10.692518 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-d0f38d35-5647-4a2d-a72f-70e463a297a3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:10.702151 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-2065e933-ffcd-4872-a305-3869ca3a4492 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:10.706747 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d0f38d35-5647-4a2d-a72f-70e463a297a3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:10.707244 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d0f38d35-5647-4a2d-a72f-70e463a297a3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:10.709943 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d0f38d35-5647-4a2d-a72f-70e463a297a3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:10.710850 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d0f38d35-5647-4a2d-a72f-70e463a297a3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:10.714872 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-2065e933-ffcd-4872-a305-3869ca3a4492 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1736 microversion: 2.1 time: 0.172255 Oct 30 01:04:10.715130 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d0f38d35-5647-4a2d-a72f-70e463a297a3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:10.715816 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d0f38d35-5647-4a2d-a72f-70e463a297a3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:10.728183 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 474/948] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:04:10 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1736 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:04:10.807179 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-d0f38d35-5647-4a2d-a72f-70e463a297a3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:10.817532 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-d0f38d35-5647-4a2d-a72f-70e463a297a3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:10.818954 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-d0f38d35-5647-4a2d-a72f-70e463a297a3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:10.820129 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-d0f38d35-5647-4a2d-a72f-70e463a297a3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1339 microversion: 2.20 time: 0.134356 Oct 30 01:04:10.820691 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 475/949] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:04:10 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1339 bytes in 136 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:04:11.743026 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-62d02b16-7197-4d33-b64b-c7a7c78c202f tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:11.743660 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-62d02b16-7197-4d33-b64b-c7a7c78c202f tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:11.750576 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-62d02b16-7197-4d33-b64b-c7a7c78c202f tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:11.751066 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-62d02b16-7197-4d33-b64b-c7a7c78c202f tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:11.751495 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-62d02b16-7197-4d33-b64b-c7a7c78c202f tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:11.752226 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-62d02b16-7197-4d33-b64b-c7a7c78c202f tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:11.752698 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-62d02b16-7197-4d33-b64b-c7a7c78c202f tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:11.753129 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-62d02b16-7197-4d33-b64b-c7a7c78c202f tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:11.847018 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-b56a1906-e524-45de-992c-37dde8ff7fdf tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:11.848284 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-b56a1906-e524-45de-992c-37dde8ff7fdf tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:11.850786 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_api:SELECT=92,nova_api:INSERT=6,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:04:11.859141 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b56a1906-e524-45de-992c-37dde8ff7fdf tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:11.860473 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b56a1906-e524-45de-992c-37dde8ff7fdf tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:11.861077 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b56a1906-e524-45de-992c-37dde8ff7fdf tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:11.862425 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b56a1906-e524-45de-992c-37dde8ff7fdf tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:11.863025 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b56a1906-e524-45de-992c-37dde8ff7fdf tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:11.863601 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b56a1906-e524-45de-992c-37dde8ff7fdf tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:11.892528 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-62d02b16-7197-4d33-b64b-c7a7c78c202f tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:11.901927 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-62d02b16-7197-4d33-b64b-c7a7c78c202f tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1736 microversion: 2.1 time: 0.162734 Oct 30 01:04:11.902750 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 475/950] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:04:11 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1736 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:04:11.998938 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-b56a1906-e524-45de-992c-37dde8ff7fdf tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:12.007368 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-b56a1906-e524-45de-992c-37dde8ff7fdf tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:12.008791 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-b56a1906-e524-45de-992c-37dde8ff7fdf tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:12.009987 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-b56a1906-e524-45de-992c-37dde8ff7fdf tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1406 microversion: 2.20 time: 0.166936 Oct 30 01:04:12.010545 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 476/951] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:04:11 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1406 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:04:12.072504 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [req-14a43829-ba84-4d1d-a5df-3d875aa1bd16 req-6f6fdd2c-c746-4ca0-b9e3-3c219c60ab6b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f4033e06-e2d3-4abb-ba25-eddfaf3e7e20", "tag": "e5d64f79-b32b-48ac-8231-fbd53262a835"}]} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:04:12.082600 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-14a43829-ba84-4d1d-a5df-3d875aa1bd16 req-6f6fdd2c-c746-4ca0-b9e3-3c219c60ab6b service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:12.083073 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-14a43829-ba84-4d1d-a5df-3d875aa1bd16 req-6f6fdd2c-c746-4ca0-b9e3-3c219c60ab6b service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:12.083497 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-14a43829-ba84-4d1d-a5df-3d875aa1bd16 req-6f6fdd2c-c746-4ca0-b9e3-3c219c60ab6b service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:12.095150 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.compute.server_external_events [req-14a43829-ba84-4d1d-a5df-3d875aa1bd16 req-6f6fdd2c-c746-4ca0-b9e3-3c219c60ab6b service nova] Creating event network-changed:e5d64f79-b32b-48ac-8231-fbd53262a835 for instance f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 on hv-cinder-maste Oct 30 01:04:12.100179 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [req-14a43829-ba84-4d1d-a5df-3d875aa1bd16 req-6f6fdd2c-c746-4ca0-b9e3-3c219c60ab6b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031471 Oct 30 01:04:12.100783 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 476/952] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:04:12 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:04:12.929871 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-be2f7f04-3036-44e2-ae84-c5d252cc1c26 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:12.930994 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-be2f7f04-3036-44e2-ae84-c5d252cc1c26 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:12.941845 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-be2f7f04-3036-44e2-ae84-c5d252cc1c26 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:12.942646 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-be2f7f04-3036-44e2-ae84-c5d252cc1c26 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:12.943361 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-be2f7f04-3036-44e2-ae84-c5d252cc1c26 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:12.944538 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-be2f7f04-3036-44e2-ae84-c5d252cc1c26 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:12.945253 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-be2f7f04-3036-44e2-ae84-c5d252cc1c26 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:12.945964 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-be2f7f04-3036-44e2-ae84-c5d252cc1c26 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:13.035937 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-f9473359-c63a-4d18-abd1-de757d145671 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:13.036625 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-f9473359-c63a-4d18-abd1-de757d145671 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:13.043532 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f9473359-c63a-4d18-abd1-de757d145671 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:13.044120 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f9473359-c63a-4d18-abd1-de757d145671 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:13.044562 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f9473359-c63a-4d18-abd1-de757d145671 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:13.045237 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f9473359-c63a-4d18-abd1-de757d145671 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:13.045680 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f9473359-c63a-4d18-abd1-de757d145671 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:13.046100 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f9473359-c63a-4d18-abd1-de757d145671 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:13.082958 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-be2f7f04-3036-44e2-ae84-c5d252cc1c26 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:13.092412 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-be2f7f04-3036-44e2-ae84-c5d252cc1c26 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1736 microversion: 2.1 time: 0.169060 Oct 30 01:04:13.092995 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 477/953] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:04:12 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1736 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:04:13.194683 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-f9473359-c63a-4d18-abd1-de757d145671 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:13.203478 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-f9473359-c63a-4d18-abd1-de757d145671 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:13.204971 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-f9473359-c63a-4d18-abd1-de757d145671 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:13.206082 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-f9473359-c63a-4d18-abd1-de757d145671 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1406 microversion: 2.20 time: 0.175115 Oct 30 01:04:13.206650 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 477/954] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:04:13 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1406 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:04:14.120929 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-0448c6da-ddf1-4258-b306-f2e01e22c384 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:14.121496 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-0448c6da-ddf1-4258-b306-f2e01e22c384 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:14.128611 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0448c6da-ddf1-4258-b306-f2e01e22c384 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:14.129097 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0448c6da-ddf1-4258-b306-f2e01e22c384 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:14.129605 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0448c6da-ddf1-4258-b306-f2e01e22c384 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:14.130407 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0448c6da-ddf1-4258-b306-f2e01e22c384 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:14.131004 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0448c6da-ddf1-4258-b306-f2e01e22c384 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:14.131612 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0448c6da-ddf1-4258-b306-f2e01e22c384 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:14.233773 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-8a25b099-16f1-4ea2-be4a-6a376180893f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:14.234352 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-8a25b099-16f1-4ea2-be4a-6a376180893f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:14.241194 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8a25b099-16f1-4ea2-be4a-6a376180893f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:14.241677 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8a25b099-16f1-4ea2-be4a-6a376180893f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:14.242129 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8a25b099-16f1-4ea2-be4a-6a376180893f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:14.242828 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8a25b099-16f1-4ea2-be4a-6a376180893f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:14.243275 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8a25b099-16f1-4ea2-be4a-6a376180893f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:14.243692 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8a25b099-16f1-4ea2-be4a-6a376180893f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:14.270202 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-0448c6da-ddf1-4258-b306-f2e01e22c384 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:14.279193 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-0448c6da-ddf1-4258-b306-f2e01e22c384 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1736 microversion: 2.1 time: 0.162464 Oct 30 01:04:14.279796 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 478/955] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:04:14 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1736 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:04:14.377285 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-8a25b099-16f1-4ea2-be4a-6a376180893f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:14.386208 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-8a25b099-16f1-4ea2-be4a-6a376180893f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:14.387630 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-8a25b099-16f1-4ea2-be4a-6a376180893f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:14.388758 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-8a25b099-16f1-4ea2-be4a-6a376180893f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1406 microversion: 2.20 time: 0.159002 Oct 30 01:04:14.389333 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 478/956] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:04:14 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1406 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:04:15.302703 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-b721184c-9673-4272-a2ec-fa6b4dd6ec07 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:15.303295 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-b721184c-9673-4272-a2ec-fa6b4dd6ec07 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:15.310504 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b721184c-9673-4272-a2ec-fa6b4dd6ec07 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:15.311018 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b721184c-9673-4272-a2ec-fa6b4dd6ec07 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:15.311458 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b721184c-9673-4272-a2ec-fa6b4dd6ec07 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:15.312201 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b721184c-9673-4272-a2ec-fa6b4dd6ec07 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:15.312710 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b721184c-9673-4272-a2ec-fa6b4dd6ec07 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:15.313085 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b721184c-9673-4272-a2ec-fa6b4dd6ec07 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:15.414478 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-996e2b8d-9cd4-4f71-97e4-80c57bb8bf8a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:15.415072 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-996e2b8d-9cd4-4f71-97e4-80c57bb8bf8a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:15.422264 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-996e2b8d-9cd4-4f71-97e4-80c57bb8bf8a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:15.422751 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-996e2b8d-9cd4-4f71-97e4-80c57bb8bf8a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:15.423202 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-996e2b8d-9cd4-4f71-97e4-80c57bb8bf8a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:15.423932 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-996e2b8d-9cd4-4f71-97e4-80c57bb8bf8a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:15.424369 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-996e2b8d-9cd4-4f71-97e4-80c57bb8bf8a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:15.424781 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-996e2b8d-9cd4-4f71-97e4-80c57bb8bf8a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:15.447690 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-b721184c-9673-4272-a2ec-fa6b4dd6ec07 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:15.456462 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-b721184c-9673-4272-a2ec-fa6b4dd6ec07 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1736 microversion: 2.1 time: 0.157713 Oct 30 01:04:15.457329 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 479/957] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:04:15 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1736 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:04:15.562728 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-996e2b8d-9cd4-4f71-97e4-80c57bb8bf8a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:15.573260 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-996e2b8d-9cd4-4f71-97e4-80c57bb8bf8a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:15.574769 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-996e2b8d-9cd4-4f71-97e4-80c57bb8bf8a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:15.575912 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-996e2b8d-9cd4-4f71-97e4-80c57bb8bf8a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1406 microversion: 2.20 time: 0.165689 Oct 30 01:04:15.576468 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 479/958] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:04:15 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1406 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 30 01:04:16.485494 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-86c59f7c-5338-498f-b887-dd3107017708 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:16.486397 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-86c59f7c-5338-498f-b887-dd3107017708 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:16.497229 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-86c59f7c-5338-498f-b887-dd3107017708 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:16.498033 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-86c59f7c-5338-498f-b887-dd3107017708 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:16.498769 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-86c59f7c-5338-498f-b887-dd3107017708 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:16.499952 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-86c59f7c-5338-498f-b887-dd3107017708 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:16.500716 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-86c59f7c-5338-498f-b887-dd3107017708 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:16.501476 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-86c59f7c-5338-498f-b887-dd3107017708 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:16.605837 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-c4705531-dab3-4427-a452-fae1b7c59f6c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:16.606558 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-c4705531-dab3-4427-a452-fae1b7c59f6c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:16.613517 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c4705531-dab3-4427-a452-fae1b7c59f6c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:16.614082 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c4705531-dab3-4427-a452-fae1b7c59f6c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:16.614585 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c4705531-dab3-4427-a452-fae1b7c59f6c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:16.615353 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c4705531-dab3-4427-a452-fae1b7c59f6c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:16.615946 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c4705531-dab3-4427-a452-fae1b7c59f6c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:16.616381 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c4705531-dab3-4427-a452-fae1b7c59f6c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:16.665750 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-86c59f7c-5338-498f-b887-dd3107017708 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:16.674308 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-86c59f7c-5338-498f-b887-dd3107017708 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1736 microversion: 2.1 time: 0.195692 Oct 30 01:04:16.674855 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 480/959] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:04:16 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1736 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:04:16.751343 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-c4705531-dab3-4427-a452-fae1b7c59f6c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:16.760190 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-c4705531-dab3-4427-a452-fae1b7c59f6c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:16.761648 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-c4705531-dab3-4427-a452-fae1b7c59f6c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:16.762763 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-c4705531-dab3-4427-a452-fae1b7c59f6c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1406 microversion: 2.20 time: 0.163793 Oct 30 01:04:16.763341 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 480/960] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:04:16 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1406 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:04:17.703148 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-52a61d84-c0e7-484a-90ca-d0fec8548698 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:17.704102 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-52a61d84-c0e7-484a-90ca-d0fec8548698 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:17.714817 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-52a61d84-c0e7-484a-90ca-d0fec8548698 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:17.715301 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-52a61d84-c0e7-484a-90ca-d0fec8548698 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:17.715759 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-52a61d84-c0e7-484a-90ca-d0fec8548698 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:17.716474 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-52a61d84-c0e7-484a-90ca-d0fec8548698 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:17.716922 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-52a61d84-c0e7-484a-90ca-d0fec8548698 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:17.717339 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-52a61d84-c0e7-484a-90ca-d0fec8548698 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:17.787737 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-09dc0cb1-450d-40a4-9b18-5f619e2a60a3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:17.788442 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-09dc0cb1-450d-40a4-9b18-5f619e2a60a3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:17.795225 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-09dc0cb1-450d-40a4-9b18-5f619e2a60a3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:17.795820 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-09dc0cb1-450d-40a4-9b18-5f619e2a60a3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:17.796321 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-09dc0cb1-450d-40a4-9b18-5f619e2a60a3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:17.796997 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-09dc0cb1-450d-40a4-9b18-5f619e2a60a3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:17.797479 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-09dc0cb1-450d-40a4-9b18-5f619e2a60a3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:17.797882 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-09dc0cb1-450d-40a4-9b18-5f619e2a60a3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:17.884723 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-52a61d84-c0e7-484a-90ca-d0fec8548698 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:17.893347 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-52a61d84-c0e7-484a-90ca-d0fec8548698 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1736 microversion: 2.1 time: 0.197769 Oct 30 01:04:17.893899 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 481/961] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:04:17 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1736 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:04:17.946429 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-09dc0cb1-450d-40a4-9b18-5f619e2a60a3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:17.957661 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-09dc0cb1-450d-40a4-9b18-5f619e2a60a3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:17.960122 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-09dc0cb1-450d-40a4-9b18-5f619e2a60a3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:17.961798 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-09dc0cb1-450d-40a4-9b18-5f619e2a60a3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1406 microversion: 2.20 time: 0.177963 Oct 30 01:04:17.962568 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 481/962] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:04:17 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1406 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:04:18.919596 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-82a36819-4c6f-46d4-b9f7-94864b3a742b tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:18.920469 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-82a36819-4c6f-46d4-b9f7-94864b3a742b tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:18.922731 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell0:SELECT=2 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:04:18.923547 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell0:SELECT=6 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:04:18.933315 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-82a36819-4c6f-46d4-b9f7-94864b3a742b tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:18.933808 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-82a36819-4c6f-46d4-b9f7-94864b3a742b tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:18.934251 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-82a36819-4c6f-46d4-b9f7-94864b3a742b tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:18.934952 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-82a36819-4c6f-46d4-b9f7-94864b3a742b tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:18.935437 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-82a36819-4c6f-46d4-b9f7-94864b3a742b tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:18.935903 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-82a36819-4c6f-46d4-b9f7-94864b3a742b tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:18.997361 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-69dd3845-1383-42ea-af57-8c8ab92de160 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:18.997913 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-69dd3845-1383-42ea-af57-8c8ab92de160 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:19.000082 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell0:SELECT=6 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:04:19.006362 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-69dd3845-1383-42ea-af57-8c8ab92de160 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:19.007908 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-69dd3845-1383-42ea-af57-8c8ab92de160 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:19.007908 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-69dd3845-1383-42ea-af57-8c8ab92de160 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:19.008252 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-69dd3845-1383-42ea-af57-8c8ab92de160 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:19.008719 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-69dd3845-1383-42ea-af57-8c8ab92de160 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:19.009149 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-69dd3845-1383-42ea-af57-8c8ab92de160 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:19.138908 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-82a36819-4c6f-46d4-b9f7-94864b3a742b tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:19.147765 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-82a36819-4c6f-46d4-b9f7-94864b3a742b tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1736 microversion: 2.1 time: 0.233987 Oct 30 01:04:19.148548 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 482/963] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:04:18 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1736 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:04:19.186791 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-69dd3845-1383-42ea-af57-8c8ab92de160 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:19.195087 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-69dd3845-1383-42ea-af57-8c8ab92de160 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:19.196534 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-69dd3845-1383-42ea-af57-8c8ab92de160 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:19.197631 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-69dd3845-1383-42ea-af57-8c8ab92de160 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1406 microversion: 2.20 time: 0.206183 Oct 30 01:04:19.198193 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 482/964] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:04:18 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1406 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:04:20.177016 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-53371500-7a12-4814-a2af-7a244a4302ba tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:20.178044 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-53371500-7a12-4814-a2af-7a244a4302ba tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:20.189516 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-53371500-7a12-4814-a2af-7a244a4302ba tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:20.190332 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-53371500-7a12-4814-a2af-7a244a4302ba tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:20.191061 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-53371500-7a12-4814-a2af-7a244a4302ba tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:20.192292 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-53371500-7a12-4814-a2af-7a244a4302ba tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:20.193051 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-53371500-7a12-4814-a2af-7a244a4302ba tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:20.193764 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-53371500-7a12-4814-a2af-7a244a4302ba tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:20.222546 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-9460f0b9-829e-419f-95aa-d1755c19bf33 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:20.223288 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-9460f0b9-829e-419f-95aa-d1755c19bf33 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:20.230198 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9460f0b9-829e-419f-95aa-d1755c19bf33 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:20.231034 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9460f0b9-829e-419f-95aa-d1755c19bf33 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:20.231532 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9460f0b9-829e-419f-95aa-d1755c19bf33 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:20.232326 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9460f0b9-829e-419f-95aa-d1755c19bf33 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:20.232835 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9460f0b9-829e-419f-95aa-d1755c19bf33 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:20.233509 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9460f0b9-829e-419f-95aa-d1755c19bf33 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:20.388280 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-53371500-7a12-4814-a2af-7a244a4302ba tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:20.396760 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-53371500-7a12-4814-a2af-7a244a4302ba tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1716 microversion: 2.1 time: 0.227274 Oct 30 01:04:20.397407 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 483/965] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:04:20 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1716 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:04:20.413489 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-9460f0b9-829e-419f-95aa-d1755c19bf33 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:20.423249 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-9460f0b9-829e-419f-95aa-d1755c19bf33 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:20.425031 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-9460f0b9-829e-419f-95aa-d1755c19bf33 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:20.426140 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-9460f0b9-829e-419f-95aa-d1755c19bf33 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1406 microversion: 2.20 time: 0.207465 Oct 30 01:04:20.426689 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 483/966] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:04:20 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1406 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:04:20.467454 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-411713f6-25bf-4ede-8209-5c8ceb89de7d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:20.467988 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-411713f6-25bf-4ede-8209-5c8ceb89de7d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:20.474310 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-411713f6-25bf-4ede-8209-5c8ceb89de7d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:20.474787 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-411713f6-25bf-4ede-8209-5c8ceb89de7d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:20.475215 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-411713f6-25bf-4ede-8209-5c8ceb89de7d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:20.475957 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-411713f6-25bf-4ede-8209-5c8ceb89de7d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:20.476405 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-411713f6-25bf-4ede-8209-5c8ceb89de7d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:20.476821 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-411713f6-25bf-4ede-8209-5c8ceb89de7d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:20.558732 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.wsgi [None req-411713f6-25bf-4ede-8209-5c8ceb89de7d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] HTTP exception thrown: Cannot 'detach_volume' instance de41579d-0173-4dba-962d-a28c85c505de while it is in vm_state rescued Oct 30 01:04:20.560372 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-411713f6-25bf-4ede-8209-5c8ceb89de7d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Request method failure captured: Oct 30 01:04:20.560372 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: request: DELETE /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de/os-volume_attachments/5eb9e9ce-78a3-49b0-b099-d73664815017 HTTP/1.1 Oct 30 01:04:20.560372 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Accept: application/json Oct 30 01:04:20.560372 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Accept-Encoding: identity Oct 30 01:04:20.560372 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Connection: close Oct 30 01:04:20.560372 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Content-Length: 0 Oct 30 01:04:20.560372 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Content-Type: application/json Oct 30 01:04:20.560372 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Host: 10.222.0.16 Oct 30 01:04:20.560372 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Openstack-System-Scope: None Oct 30 01:04:20.560372 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: User-Agent: python-urllib3/1.26.12 Oct 30 01:04:20.560372 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Auth-Token: gAAAAABjXcwtPy0eLS3MC3-cmfmCjFSfeavovRIYXfEfNVAywVbIQ30ctEe5V4sGvi1A3M4sDeUVrGvF9i6stAWwO3TejIedmS8RLhBtx-h_JsINfLnyH884rCMd4-euC7isoMsjNUbUBUpUbdHwx6JXx5oDJcwXMr1RZh9X14AJM7k_e6z6bu8 Oct 30 01:04:20.560372 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Domain-Id: None Oct 30 01:04:20.560372 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Domain-Name: None Oct 30 01:04:20.560372 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Identity-Status: Confirmed Oct 30 01:04:20.560372 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Is-Admin-Project: True Oct 30 01:04:20.560372 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Domain-Id: default Oct 30 01:04:20.560372 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Domain-Name: Default Oct 30 01:04:20.560372 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Id: f48276ac51dd4bb1912fc670433cf6c4 Oct 30 01:04:20.560372 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-764720217 Oct 30 01:04:20.560372 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Role: reader,member Oct 30 01:04:20.560372 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Roles: reader,member Oct 30 01:04:20.561791 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_f48276ac51dd4bb1912fc670433cf6c4"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f48276ac51dd4bb1912fc670433cf6c4"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/f48276ac51dd4bb1912fc670433cf6c4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f48276ac51dd4bb1912fc670433cf6c4"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 01:04:20.561791 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant: tempest-ServerRescueNegativeTestJSON-764720217 Oct 30 01:04:20.561791 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant-Id: f48276ac51dd4bb1912fc670433cf6c4 Oct 30 01:04:20.561791 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-764720217 Oct 30 01:04:20.561791 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User: tempest-ServerRescueNegativeTestJSON-764720217-project Oct 30 01:04:20.561791 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Domain-Id: default Oct 30 01:04:20.561791 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Domain-Name: Default Oct 30 01:04:20.561791 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Id: 31b7342ecf07408cab6bb040f361fa1f Oct 30 01:04:20.561791 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Name: tempest-ServerRescueNegativeTestJSON-764720217-project Oct 30 01:04:20.561791 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: method: > Oct 30 01:04:20.561791 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: exception: Cannot 'detach_volume' instance de41579d-0173-4dba-962d-a28c85c505de while it is in vm_state rescued Oct 30 01:04:20.562519 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: action_args: {'server_id': 'de41579d-0173-4dba-962d-a28c85c505de', 'id': '5eb9e9ce-78a3-49b0-b099-d73664815017'} Oct 30 01:04:20.562519 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 01:04:20.562519 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:04:20.562519 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Oct 30 01:04:20.562519 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Oct 30 01:04:20.562519 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 30 01:04:20.562519 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 30 01:04:20.562519 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Oct 30 01:04:20.562519 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Oct 30 01:04:20.562519 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance de41579d-0173-4dba-962d-a28c85c505de in vm_state rescued. Cannot detach_volume while the instance is in this state. Oct 30 01:04:20.562519 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:04:20.562519 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:04:20.562519 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:04:20.562519 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:04:20.562519 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:04:20.562519 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:04:20.562519 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 01:04:20.562519 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 01:04:20.562519 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 01:04:20.562519 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 01:04:20.564016 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 573, in delete Oct 30 01:04:20.564016 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Oct 30 01:04:20.564016 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Oct 30 01:04:20.564016 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Oct 30 01:04:20.564016 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'detach_volume' instance de41579d-0173-4dba-962d-a28c85c505de while it is in vm_state rescued Oct 30 01:04:20.564016 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:04:20.564016 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:04:20.564016 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:04:20.564016 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:04:20.564016 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:04:20.564016 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:04:20.564016 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 01:04:20.564016 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 01:04:20.564016 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'detach_volume' instance de41579d-0173-4dba-962d-a28c85c505de while it is in vm_state rescued Oct 30 01:04:20.564016 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:04:20.564016 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-411713f6-25bf-4ede-8209-5c8ceb89de7d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Returning 409 to user: Cannot 'detach_volume' instance de41579d-0173-4dba-962d-a28c85c505de while it is in vm_state rescued {{(pid=118961) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 01:04:20.565026 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-411713f6-25bf-4ede-8209-5c8ceb89de7d tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "DELETE /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de/os-volume_attachments/5eb9e9ce-78a3-49b0-b099-d73664815017" status: 409 len: 152 microversion: 2.1 time: 0.099144 Oct 30 01:04:20.565026 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 484/967] 10.222.0.16 () {58 vars in 1416 bytes} [Sun Oct 30 01:04:20 2022] DELETE /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de/os-volume_attachments/5eb9e9ce-78a3-49b0-b099-d73664815017 => generated 152 bytes in 101 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Oct 30 01:04:20.570832 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-9be5d920-f1a6-4b7e-8568-4e772d6251d5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:04:20.571366 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-9be5d920-f1a6-4b7e-8568-4e772d6251d5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:20.577927 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9be5d920-f1a6-4b7e-8568-4e772d6251d5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:20.578408 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9be5d920-f1a6-4b7e-8568-4e772d6251d5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:20.578836 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9be5d920-f1a6-4b7e-8568-4e772d6251d5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:20.579525 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9be5d920-f1a6-4b7e-8568-4e772d6251d5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:20.580060 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9be5d920-f1a6-4b7e-8568-4e772d6251d5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:20.580510 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9be5d920-f1a6-4b7e-8568-4e772d6251d5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:20.652813 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-9be5d920-f1a6-4b7e-8568-4e772d6251d5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "POST /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de/action" status: 202 len: 0 microversion: 2.1 time: 0.085660 Oct 30 01:04:20.653467 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 484/968] 10.222.0.16 () {60 vars in 1278 bytes} [Sun Oct 30 01:04:20 2022] POST /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de/action => generated 0 bytes in 87 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 30 01:04:20.664096 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-08f7c236-351a-42f6-8a44-9ab2e00f1f49 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:20.664716 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-08f7c236-351a-42f6-8a44-9ab2e00f1f49 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:20.670904 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-08f7c236-351a-42f6-8a44-9ab2e00f1f49 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:20.671398 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-08f7c236-351a-42f6-8a44-9ab2e00f1f49 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:20.671869 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-08f7c236-351a-42f6-8a44-9ab2e00f1f49 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:20.672589 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-08f7c236-351a-42f6-8a44-9ab2e00f1f49 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:20.673029 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-08f7c236-351a-42f6-8a44-9ab2e00f1f49 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:20.673527 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-08f7c236-351a-42f6-8a44-9ab2e00f1f49 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:20.813972 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-08f7c236-351a-42f6-8a44-9ab2e00f1f49 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:20.822427 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-08f7c236-351a-42f6-8a44-9ab2e00f1f49 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1724 microversion: 2.1 time: 0.164153 Oct 30 01:04:20.822963 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 485/969] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:04:20 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1724 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:04:21.457074 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-1b45e090-2bd1-473d-8221-42871898ad67 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:21.458705 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-1b45e090-2bd1-473d-8221-42871898ad67 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:21.467609 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1b45e090-2bd1-473d-8221-42871898ad67 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:21.468048 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1b45e090-2bd1-473d-8221-42871898ad67 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:21.468474 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1b45e090-2bd1-473d-8221-42871898ad67 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:21.469155 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1b45e090-2bd1-473d-8221-42871898ad67 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:21.469598 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1b45e090-2bd1-473d-8221-42871898ad67 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:21.470013 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1b45e090-2bd1-473d-8221-42871898ad67 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:21.614474 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-1b45e090-2bd1-473d-8221-42871898ad67 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:21.622931 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-1b45e090-2bd1-473d-8221-42871898ad67 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:21.624587 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-1b45e090-2bd1-473d-8221-42871898ad67 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:21.625697 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-1b45e090-2bd1-473d-8221-42871898ad67 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1406 microversion: 2.20 time: 0.176595 Oct 30 01:04:21.626255 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 485/970] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:04:21 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1406 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:04:21.696547 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [req-9f8d1203-d695-4a1f-b198-2b3aef46ff59 req-571263dc-c56f-426e-a470-b0955776b690 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f4033e06-e2d3-4abb-ba25-eddfaf3e7e20", "name": "network-vif-plugged", "status": "completed", "tag": "e5d64f79-b32b-48ac-8231-fbd53262a835"}]} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:04:21.707283 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-9f8d1203-d695-4a1f-b198-2b3aef46ff59 req-571263dc-c56f-426e-a470-b0955776b690 service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:21.707798 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-9f8d1203-d695-4a1f-b198-2b3aef46ff59 req-571263dc-c56f-426e-a470-b0955776b690 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:21.708520 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-9f8d1203-d695-4a1f-b198-2b3aef46ff59 req-571263dc-c56f-426e-a470-b0955776b690 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:21.721354 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.compute.server_external_events [req-9f8d1203-d695-4a1f-b198-2b3aef46ff59 req-571263dc-c56f-426e-a470-b0955776b690 service nova] Creating event network-vif-plugged:e5d64f79-b32b-48ac-8231-fbd53262a835 for instance f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 on hv-cinder-maste Oct 30 01:04:21.726398 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [req-9f8d1203-d695-4a1f-b198-2b3aef46ff59 req-571263dc-c56f-426e-a470-b0955776b690 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033632 Oct 30 01:04:21.727059 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 486/971] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:04:21 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:04:21.845746 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-86662b9f-1daf-45fc-bbe8-1f03ddc32183 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:21.846362 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-86662b9f-1daf-45fc-bbe8-1f03ddc32183 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:21.852786 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-86662b9f-1daf-45fc-bbe8-1f03ddc32183 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:21.853269 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-86662b9f-1daf-45fc-bbe8-1f03ddc32183 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:21.853704 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-86662b9f-1daf-45fc-bbe8-1f03ddc32183 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:21.854374 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-86662b9f-1daf-45fc-bbe8-1f03ddc32183 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:21.854892 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-86662b9f-1daf-45fc-bbe8-1f03ddc32183 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:21.855325 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-86662b9f-1daf-45fc-bbe8-1f03ddc32183 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:21.859421 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:04:21.993347 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-86662b9f-1daf-45fc-bbe8-1f03ddc32183 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:22.001938 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-86662b9f-1daf-45fc-bbe8-1f03ddc32183 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1724 microversion: 2.1 time: 0.160131 Oct 30 01:04:22.002527 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 486/972] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:04:21 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1724 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:04:22.655815 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-fd971fe4-de04-46ec-a26b-4e4331a3a587 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:22.656869 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-fd971fe4-de04-46ec-a26b-4e4331a3a587 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:22.667422 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-fd971fe4-de04-46ec-a26b-4e4331a3a587 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:22.667934 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-fd971fe4-de04-46ec-a26b-4e4331a3a587 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:22.668402 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-fd971fe4-de04-46ec-a26b-4e4331a3a587 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:22.669105 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-fd971fe4-de04-46ec-a26b-4e4331a3a587 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:22.669551 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-fd971fe4-de04-46ec-a26b-4e4331a3a587 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:22.669966 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-fd971fe4-de04-46ec-a26b-4e4331a3a587 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:22.817855 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-fd971fe4-de04-46ec-a26b-4e4331a3a587 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:22.829995 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-fd971fe4-de04-46ec-a26b-4e4331a3a587 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:22.832275 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-fd971fe4-de04-46ec-a26b-4e4331a3a587 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:22.833973 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-fd971fe4-de04-46ec-a26b-4e4331a3a587 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1406 microversion: 2.20 time: 0.185279 Oct 30 01:04:22.835113 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 487/973] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:04:22 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1406 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:04:23.029050 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-dcf5982c-f22b-4aa8-a106-c982ead5c323 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:23.029730 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-dcf5982c-f22b-4aa8-a106-c982ead5c323 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:23.037660 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dcf5982c-f22b-4aa8-a106-c982ead5c323 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:23.038256 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dcf5982c-f22b-4aa8-a106-c982ead5c323 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:23.038744 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dcf5982c-f22b-4aa8-a106-c982ead5c323 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:23.039604 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dcf5982c-f22b-4aa8-a106-c982ead5c323 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:23.040174 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dcf5982c-f22b-4aa8-a106-c982ead5c323 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:23.040678 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dcf5982c-f22b-4aa8-a106-c982ead5c323 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:23.181598 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-dcf5982c-f22b-4aa8-a106-c982ead5c323 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:23.190182 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-dcf5982c-f22b-4aa8-a106-c982ead5c323 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1724 microversion: 2.1 time: 0.167326 Oct 30 01:04:23.190763 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 487/974] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:04:23 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1724 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:04:23.864464 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-6d2e71bb-9e44-4957-8961-df41d3695e7f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:23.865431 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-6d2e71bb-9e44-4957-8961-df41d3695e7f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:23.876571 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6d2e71bb-9e44-4957-8961-df41d3695e7f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:23.877373 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6d2e71bb-9e44-4957-8961-df41d3695e7f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:23.878084 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6d2e71bb-9e44-4957-8961-df41d3695e7f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:23.879234 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6d2e71bb-9e44-4957-8961-df41d3695e7f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:23.880034 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6d2e71bb-9e44-4957-8961-df41d3695e7f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:23.880771 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6d2e71bb-9e44-4957-8961-df41d3695e7f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:24.020899 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-6d2e71bb-9e44-4957-8961-df41d3695e7f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:24.034017 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-6d2e71bb-9e44-4957-8961-df41d3695e7f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:24.036417 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-6d2e71bb-9e44-4957-8961-df41d3695e7f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:24.038120 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-6d2e71bb-9e44-4957-8961-df41d3695e7f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1406 microversion: 2.20 time: 0.180156 Oct 30 01:04:24.038956 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 488/975] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:04:23 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1406 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:04:24.215808 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-d711b13b-3ebc-4718-8f53-071bc8d45c06 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:24.216398 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-d711b13b-3ebc-4718-8f53-071bc8d45c06 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:24.224471 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d711b13b-3ebc-4718-8f53-071bc8d45c06 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:24.224949 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d711b13b-3ebc-4718-8f53-071bc8d45c06 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:24.225360 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d711b13b-3ebc-4718-8f53-071bc8d45c06 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:24.226060 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d711b13b-3ebc-4718-8f53-071bc8d45c06 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:24.226503 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d711b13b-3ebc-4718-8f53-071bc8d45c06 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:24.226920 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d711b13b-3ebc-4718-8f53-071bc8d45c06 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:24.360602 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-d711b13b-3ebc-4718-8f53-071bc8d45c06 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:24.374355 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-d711b13b-3ebc-4718-8f53-071bc8d45c06 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1724 microversion: 2.1 time: 0.163315 Oct 30 01:04:24.374937 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 488/976] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:04:24 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1724 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:04:25.063201 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-00f3ffec-666e-40f8-9826-6a7cd4c7408e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:25.063838 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-00f3ffec-666e-40f8-9826-6a7cd4c7408e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:25.070307 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-00f3ffec-666e-40f8-9826-6a7cd4c7408e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:25.070808 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-00f3ffec-666e-40f8-9826-6a7cd4c7408e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:25.071231 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-00f3ffec-666e-40f8-9826-6a7cd4c7408e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:25.071977 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-00f3ffec-666e-40f8-9826-6a7cd4c7408e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:25.072460 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-00f3ffec-666e-40f8-9826-6a7cd4c7408e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:25.072873 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-00f3ffec-666e-40f8-9826-6a7cd4c7408e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:25.210975 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-00f3ffec-666e-40f8-9826-6a7cd4c7408e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:25.220456 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-00f3ffec-666e-40f8-9826-6a7cd4c7408e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:25.221875 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-00f3ffec-666e-40f8-9826-6a7cd4c7408e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:25.222991 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-00f3ffec-666e-40f8-9826-6a7cd4c7408e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1406 microversion: 2.20 time: 0.163872 Oct 30 01:04:25.223527 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 489/977] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:04:25 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1406 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:04:25.397999 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-61ff949b-ae03-4651-a090-decbec443913 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:25.398617 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-61ff949b-ae03-4651-a090-decbec443913 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:25.405451 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-61ff949b-ae03-4651-a090-decbec443913 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:25.405938 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-61ff949b-ae03-4651-a090-decbec443913 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:25.406371 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-61ff949b-ae03-4651-a090-decbec443913 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:25.407093 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-61ff949b-ae03-4651-a090-decbec443913 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:25.407545 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-61ff949b-ae03-4651-a090-decbec443913 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:25.407995 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-61ff949b-ae03-4651-a090-decbec443913 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:25.547508 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-61ff949b-ae03-4651-a090-decbec443913 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:25.558152 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-61ff949b-ae03-4651-a090-decbec443913 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1730 microversion: 2.1 time: 0.163833 Oct 30 01:04:25.559089 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 489/978] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:04:25 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1730 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:04:25.587510 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-bb8319d0-e36f-47ec-bc48-ace7d3cda548 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:25.588030 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-bb8319d0-e36f-47ec-bc48-ace7d3cda548 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:25.594582 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bb8319d0-e36f-47ec-bc48-ace7d3cda548 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:25.595065 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bb8319d0-e36f-47ec-bc48-ace7d3cda548 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:25.595522 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bb8319d0-e36f-47ec-bc48-ace7d3cda548 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:25.596244 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bb8319d0-e36f-47ec-bc48-ace7d3cda548 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:25.596690 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bb8319d0-e36f-47ec-bc48-ace7d3cda548 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:25.597104 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bb8319d0-e36f-47ec-bc48-ace7d3cda548 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:25.676008 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-bb8319d0-e36f-47ec-bc48-ace7d3cda548 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.092716 Oct 30 01:04:25.677337 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 490/979] 10.222.0.16 () {58 vars in 1275 bytes} [Sun Oct 30 01:04:25 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de/os-interface => generated 284 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:04:26.247849 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-dce6aae1-5c6f-4054-bcb9-4bc6a1eeab70 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:26.248451 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-dce6aae1-5c6f-4054-bcb9-4bc6a1eeab70 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:26.250324 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_api:SELECT=91,nova_api:INSERT=1 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:04:26.255652 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dce6aae1-5c6f-4054-bcb9-4bc6a1eeab70 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:26.256145 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dce6aae1-5c6f-4054-bcb9-4bc6a1eeab70 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:26.256568 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dce6aae1-5c6f-4054-bcb9-4bc6a1eeab70 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:26.257255 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dce6aae1-5c6f-4054-bcb9-4bc6a1eeab70 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:26.257696 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dce6aae1-5c6f-4054-bcb9-4bc6a1eeab70 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:26.258103 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dce6aae1-5c6f-4054-bcb9-4bc6a1eeab70 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:26.506522 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-dce6aae1-5c6f-4054-bcb9-4bc6a1eeab70 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:26.515248 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-dce6aae1-5c6f-4054-bcb9-4bc6a1eeab70 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:26.517055 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-dce6aae1-5c6f-4054-bcb9-4bc6a1eeab70 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:26.518638 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-dce6aae1-5c6f-4054-bcb9-4bc6a1eeab70 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1406 microversion: 2.20 time: 0.274812 Oct 30 01:04:26.520871 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 490/980] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:04:26 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1406 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:04:26.895703 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [req-9636b03c-9e6a-4c4b-aeb4-aabf266a23a0 req-1bdfdd6a-06f1-4711-b73c-450fb58eee78 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "de41579d-0173-4dba-962d-a28c85c505de", "tag": "c4782435-5ac9-430e-95c9-c061a2747b76"}]} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:04:26.906608 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-9636b03c-9e6a-4c4b-aeb4-aabf266a23a0 req-1bdfdd6a-06f1-4711-b73c-450fb58eee78 service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:26.906839 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-9636b03c-9e6a-4c4b-aeb4-aabf266a23a0 req-1bdfdd6a-06f1-4711-b73c-450fb58eee78 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:26.907277 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-9636b03c-9e6a-4c4b-aeb4-aabf266a23a0 req-1bdfdd6a-06f1-4711-b73c-450fb58eee78 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:26.919234 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.compute.server_external_events [req-9636b03c-9e6a-4c4b-aeb4-aabf266a23a0 req-1bdfdd6a-06f1-4711-b73c-450fb58eee78 service nova] Creating event network-changed:c4782435-5ac9-430e-95c9-c061a2747b76 for instance de41579d-0173-4dba-962d-a28c85c505de on hv-cinder-maste Oct 30 01:04:26.925271 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [req-9636b03c-9e6a-4c4b-aeb4-aabf266a23a0 req-1bdfdd6a-06f1-4711-b73c-450fb58eee78 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033143 Oct 30 01:04:26.925597 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 491/981] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:04:26 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:04:27.546601 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-d2148bba-753c-4209-a388-c1dc8db8e310 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:27.547215 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-d2148bba-753c-4209-a388-c1dc8db8e310 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:27.554058 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d2148bba-753c-4209-a388-c1dc8db8e310 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:27.554581 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d2148bba-753c-4209-a388-c1dc8db8e310 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:27.555060 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d2148bba-753c-4209-a388-c1dc8db8e310 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:27.555698 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d2148bba-753c-4209-a388-c1dc8db8e310 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:27.556171 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d2148bba-753c-4209-a388-c1dc8db8e310 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:27.556590 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d2148bba-753c-4209-a388-c1dc8db8e310 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:27.694828 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-d2148bba-753c-4209-a388-c1dc8db8e310 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:27.703965 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-d2148bba-753c-4209-a388-c1dc8db8e310 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:27.705384 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-d2148bba-753c-4209-a388-c1dc8db8e310 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:27.706556 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-d2148bba-753c-4209-a388-c1dc8db8e310 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1598 microversion: 2.20 time: 0.164646 Oct 30 01:04:27.707107 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 491/982] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:04:27 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1598 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:04:27.732430 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-b89055ac-1496-4918-9907-0f6dd629e7fb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:27.732984 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-b89055ac-1496-4918-9907-0f6dd629e7fb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:27.739695 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b89055ac-1496-4918-9907-0f6dd629e7fb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:27.740208 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b89055ac-1496-4918-9907-0f6dd629e7fb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:27.740659 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b89055ac-1496-4918-9907-0f6dd629e7fb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:27.741358 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b89055ac-1496-4918-9907-0f6dd629e7fb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:27.741825 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b89055ac-1496-4918-9907-0f6dd629e7fb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:27.742235 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b89055ac-1496-4918-9907-0f6dd629e7fb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:27.759059 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell1:SELECT=100 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:04:27.820836 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-b89055ac-1496-4918-9907-0f6dd629e7fb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20/os-interface" status: 200 len: 285 microversion: 2.20 time: 0.092957 Oct 30 01:04:27.821454 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 492/983] 10.222.0.16 () {60 vars in 1316 bytes} [Sun Oct 30 01:04:27 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20/os-interface => generated 285 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 30 01:04:28.940021 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [req-18a8a25e-dcfb-42ff-b95a-c7cbc9431cfd req-1ccb53e7-01b4-4c64-82d7-9ee54f4faaf6 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "de41579d-0173-4dba-962d-a28c85c505de", "tag": "c4782435-5ac9-430e-95c9-c061a2747b76"}, {"name": "network-changed", "server_uuid": "f4033e06-e2d3-4abb-ba25-eddfaf3e7e20", "tag": "e5d64f79-b32b-48ac-8231-fbd53262a835"}]} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:04:28.951156 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-18a8a25e-dcfb-42ff-b95a-c7cbc9431cfd req-1ccb53e7-01b4-4c64-82d7-9ee54f4faaf6 service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:28.951624 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-18a8a25e-dcfb-42ff-b95a-c7cbc9431cfd req-1ccb53e7-01b4-4c64-82d7-9ee54f4faaf6 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:28.952071 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-18a8a25e-dcfb-42ff-b95a-c7cbc9431cfd req-1ccb53e7-01b4-4c64-82d7-9ee54f4faaf6 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:28.965021 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.compute.server_external_events [req-18a8a25e-dcfb-42ff-b95a-c7cbc9431cfd req-1ccb53e7-01b4-4c64-82d7-9ee54f4faaf6 service nova] Creating event network-changed:c4782435-5ac9-430e-95c9-c061a2747b76 for instance de41579d-0173-4dba-962d-a28c85c505de on hv-cinder-maste Oct 30 01:04:28.965515 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.compute.server_external_events [req-18a8a25e-dcfb-42ff-b95a-c7cbc9431cfd req-1ccb53e7-01b4-4c64-82d7-9ee54f4faaf6 service nova] Creating event network-changed:e5d64f79-b32b-48ac-8231-fbd53262a835 for instance f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 on hv-cinder-maste Oct 30 01:04:28.972133 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [req-18a8a25e-dcfb-42ff-b95a-c7cbc9431cfd req-1ccb53e7-01b4-4c64-82d7-9ee54f4faaf6 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 346 microversion: 2.1 time: 0.035875 Oct 30 01:04:28.972924 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 492/984] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:04:28 2022] POST /compute/v2.1/os-server-external-events => generated 346 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:04:44.787616 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_api:SELECT=33 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:04:44.792279 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-fd16ebde-38dc-4efe-b2fd-3c7f7782448b tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:44.794455 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-fd16ebde-38dc-4efe-b2fd-3c7f7782448b tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:44.801333 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-fd16ebde-38dc-4efe-b2fd-3c7f7782448b tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:44.801818 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-fd16ebde-38dc-4efe-b2fd-3c7f7782448b tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:44.802249 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-fd16ebde-38dc-4efe-b2fd-3c7f7782448b tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:44.802941 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-fd16ebde-38dc-4efe-b2fd-3c7f7782448b tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:44.803400 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-fd16ebde-38dc-4efe-b2fd-3c7f7782448b tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:44.803855 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-fd16ebde-38dc-4efe-b2fd-3c7f7782448b tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:44.833722 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-fd16ebde-38dc-4efe-b2fd-3c7f7782448b tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.047899 Oct 30 01:04:44.834279 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 493/985] 10.222.0.16 () {58 vars in 1302 bytes} [Sun Oct 30 01:04:44 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14/os-volume_attachments => generated 25 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 30 01:04:44.905262 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_api:SELECT=5 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:04:44.906290 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell1:SELECT=36 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:04:44.910771 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-c35f46c1-cd86-4e2f-8351-231e1fc9dfa0 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:44.911346 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-c35f46c1-cd86-4e2f-8351-231e1fc9dfa0 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:44.918302 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c35f46c1-cd86-4e2f-8351-231e1fc9dfa0 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:44.918773 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c35f46c1-cd86-4e2f-8351-231e1fc9dfa0 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:44.919193 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c35f46c1-cd86-4e2f-8351-231e1fc9dfa0 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:44.919895 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c35f46c1-cd86-4e2f-8351-231e1fc9dfa0 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:44.920332 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c35f46c1-cd86-4e2f-8351-231e1fc9dfa0 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:44.920742 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c35f46c1-cd86-4e2f-8351-231e1fc9dfa0 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:44.948195 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-c35f46c1-cd86-4e2f-8351-231e1fc9dfa0 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Going to try to terminate instance {{(pid=118962) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 30 01:04:45.032195 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-c35f46c1-cd86-4e2f-8351-231e1fc9dfa0 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "DELETE /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 204 len: 0 microversion: 2.1 time: 0.128707 Oct 30 01:04:45.033128 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 493/986] 10.222.0.16 () {58 vars in 1239 bytes} [Sun Oct 30 01:04:44 2022] DELETE /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 30 01:04:45.042794 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-f1f8b053-cc2a-49e2-b3b2-33f6d99c3f5c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:45.043494 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-f1f8b053-cc2a-49e2-b3b2-33f6d99c3f5c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:45.049537 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f1f8b053-cc2a-49e2-b3b2-33f6d99c3f5c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:45.050011 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f1f8b053-cc2a-49e2-b3b2-33f6d99c3f5c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:45.050444 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f1f8b053-cc2a-49e2-b3b2-33f6d99c3f5c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:45.051159 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f1f8b053-cc2a-49e2-b3b2-33f6d99c3f5c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:45.051604 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f1f8b053-cc2a-49e2-b3b2-33f6d99c3f5c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:45.052048 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f1f8b053-cc2a-49e2-b3b2-33f6d99c3f5c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:45.186088 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-f1f8b053-cc2a-49e2-b3b2-33f6d99c3f5c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:45.194179 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-f1f8b053-cc2a-49e2-b3b2-33f6d99c3f5c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1674 microversion: 2.1 time: 0.155307 Oct 30 01:04:45.194757 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 494/987] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:04:45 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1674 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 01:04:46.219230 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-b45727a1-be23-4653-95dc-98ca6e732e07 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:46.219861 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-b45727a1-be23-4653-95dc-98ca6e732e07 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:46.226760 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b45727a1-be23-4653-95dc-98ca6e732e07 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:46.227248 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b45727a1-be23-4653-95dc-98ca6e732e07 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:46.227677 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b45727a1-be23-4653-95dc-98ca6e732e07 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:46.228392 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b45727a1-be23-4653-95dc-98ca6e732e07 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:46.228833 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b45727a1-be23-4653-95dc-98ca6e732e07 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:46.229245 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b45727a1-be23-4653-95dc-98ca6e732e07 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:46.391214 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-b45727a1-be23-4653-95dc-98ca6e732e07 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:46.402966 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-b45727a1-be23-4653-95dc-98ca6e732e07 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1674 microversion: 2.1 time: 0.188575 Oct 30 01:04:46.403548 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 494/988] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:04:46 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1674 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:04:47.429207 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-ad8e913b-b63d-4f85-97fb-ff6ee826eb61 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:47.429846 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-ad8e913b-b63d-4f85-97fb-ff6ee826eb61 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:47.436798 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ad8e913b-b63d-4f85-97fb-ff6ee826eb61 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:47.437283 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ad8e913b-b63d-4f85-97fb-ff6ee826eb61 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:47.437717 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ad8e913b-b63d-4f85-97fb-ff6ee826eb61 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:47.438404 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ad8e913b-b63d-4f85-97fb-ff6ee826eb61 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:47.438852 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ad8e913b-b63d-4f85-97fb-ff6ee826eb61 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:47.439370 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ad8e913b-b63d-4f85-97fb-ff6ee826eb61 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:47.511423 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-ad8e913b-b63d-4f85-97fb-ff6ee826eb61 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:47.519356 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-ad8e913b-b63d-4f85-97fb-ff6ee826eb61 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1606 microversion: 2.1 time: 0.094168 Oct 30 01:04:47.520232 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 495/989] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:04:47 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1606 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:04:47.606961 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [req-d3dbd422-eb00-41a5-80dd-29dd2aba9203 req-830248fe-9358-410e-b2c5-5c8673173421 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "70ad3ec1-64bc-40ac-bb98-cc82d19e2a14", "name": "network-vif-deleted", "tag": "dcb64cf9-6d5f-470b-8042-0cdbf3e91991"}]} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:04:47.619449 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-d3dbd422-eb00-41a5-80dd-29dd2aba9203 req-830248fe-9358-410e-b2c5-5c8673173421 service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:47.620062 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-d3dbd422-eb00-41a5-80dd-29dd2aba9203 req-830248fe-9358-410e-b2c5-5c8673173421 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:47.620571 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-d3dbd422-eb00-41a5-80dd-29dd2aba9203 req-830248fe-9358-410e-b2c5-5c8673173421 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:47.637844 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.compute.server_external_events [req-d3dbd422-eb00-41a5-80dd-29dd2aba9203 req-830248fe-9358-410e-b2c5-5c8673173421 service nova] Creating event network-vif-deleted:dcb64cf9-6d5f-470b-8042-0cdbf3e91991 for instance 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 on hv-cinder-maste Oct 30 01:04:47.644030 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [req-d3dbd422-eb00-41a5-80dd-29dd2aba9203 req-830248fe-9358-410e-b2c5-5c8673173421 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040846 Oct 30 01:04:47.648563 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 495/990] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:04:47 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:04:48.543609 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-2d2637da-9423-4da1-bc2e-35a65b99318c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:48.544305 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-2d2637da-9423-4da1-bc2e-35a65b99318c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:48.551461 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2d2637da-9423-4da1-bc2e-35a65b99318c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:48.551979 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2d2637da-9423-4da1-bc2e-35a65b99318c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:48.552408 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2d2637da-9423-4da1-bc2e-35a65b99318c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:48.553114 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2d2637da-9423-4da1-bc2e-35a65b99318c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:48.553556 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2d2637da-9423-4da1-bc2e-35a65b99318c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:48.553974 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2d2637da-9423-4da1-bc2e-35a65b99318c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:48.586396 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2d2637da-9423-4da1-bc2e-35a65b99318c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:48.586885 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2d2637da-9423-4da1-bc2e-35a65b99318c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:48.587310 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2d2637da-9423-4da1-bc2e-35a65b99318c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:48.587780 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.objects.instance [None req-2d2637da-9423-4da1-bc2e-35a65b99318c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lazy-loading 'fault' on Instance uuid 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 {{(pid=118961) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 30 01:04:48.629368 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-2d2637da-9423-4da1-bc2e-35a65b99318c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:48.637500 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-2d2637da-9423-4da1-bc2e-35a65b99318c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 200 len: 1328 microversion: 2.1 time: 0.098567 Oct 30 01:04:48.638075 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 496/991] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:04:48 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 1328 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:04:49.664844 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-b21c6d09-7458-4459-a64e-0d9abacc2eeb tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:49.665467 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-b21c6d09-7458-4459-a64e-0d9abacc2eeb tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:49.672398 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b21c6d09-7458-4459-a64e-0d9abacc2eeb tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:49.672899 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b21c6d09-7458-4459-a64e-0d9abacc2eeb tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:49.673344 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b21c6d09-7458-4459-a64e-0d9abacc2eeb tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:49.674065 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b21c6d09-7458-4459-a64e-0d9abacc2eeb tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:49.674533 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b21c6d09-7458-4459-a64e-0d9abacc2eeb tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:49.674964 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b21c6d09-7458-4459-a64e-0d9abacc2eeb tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:49.689170 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.wsgi [None req-b21c6d09-7458-4459-a64e-0d9abacc2eeb tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] HTTP exception thrown: Instance 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 could not be found. Oct 30 01:04:49.690852 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-b21c6d09-7458-4459-a64e-0d9abacc2eeb tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Request method failure captured: Oct 30 01:04:49.690852 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: request: GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 HTTP/1.1 Oct 30 01:04:49.690852 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept: application/json Oct 30 01:04:49.690852 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept-Encoding: identity Oct 30 01:04:49.690852 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Connection: close Oct 30 01:04:49.690852 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Length: 0 Oct 30 01:04:49.690852 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Type: application/json Oct 30 01:04:49.690852 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Host: 10.222.0.16 Oct 30 01:04:49.690852 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Openstack-System-Scope: None Oct 30 01:04:49.690852 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: User-Agent: python-urllib3/1.26.12 Oct 30 01:04:49.690852 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Auth-Token: gAAAAABjXcufJywtdFM5PObVwvc4XfvnADFZy4zohCugYTCT6kU4WYIm9tEeU-k9LqQ38owLu40IDCuTtQtYT30Plcr8MKntgwthetV3Q2djsPe7ErJwj43_xWiRbN29t1NWovB4v2ZaEkG-yf_zjlv7AkC5WmwDZXliUm7DodcvaVfundKFDhc Oct 30 01:04:49.690852 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Id: None Oct 30 01:04:49.690852 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Name: None Oct 30 01:04:49.690852 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Identity-Status: Confirmed Oct 30 01:04:49.690852 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Is-Admin-Project: True Oct 30 01:04:49.690852 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Id: default Oct 30 01:04:49.690852 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Name: Default Oct 30 01:04:49.690852 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Id: 8bf407954ed34ebfaf56db206df4fc84 Oct 30 01:04:49.690852 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Name: tempest-AttachVolumeTestJSON-298540627 Oct 30 01:04:49.690852 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Role: reader,member Oct 30 01:04:49.690852 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Roles: reader,member Oct 30 01:04:49.692504 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_8bf407954ed34ebfaf56db206df4fc84"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8bf407954ed34ebfaf56db206df4fc84"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/8bf407954ed34ebfaf56db206df4fc84"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8bf407954ed34ebfaf56db206df4fc84"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 01:04:49.692504 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant: tempest-AttachVolumeTestJSON-298540627 Oct 30 01:04:49.692504 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Id: 8bf407954ed34ebfaf56db206df4fc84 Oct 30 01:04:49.692504 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Name: tempest-AttachVolumeTestJSON-298540627 Oct 30 01:04:49.692504 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User: tempest-AttachVolumeTestJSON-298540627-project Oct 30 01:04:49.692504 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Id: default Oct 30 01:04:49.692504 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Name: Default Oct 30 01:04:49.692504 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Id: 1c0e5bd11ac14259ad376ce54bcb7025 Oct 30 01:04:49.692504 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Name: tempest-AttachVolumeTestJSON-298540627-project Oct 30 01:04:49.692504 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: method: > Oct 30 01:04:49.692504 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: exception: Instance 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 could not be found. Oct 30 01:04:49.692504 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: action_args: {'id': '70ad3ec1-64bc-40ac-bb98-cc82d19e2a14'} Oct 30 01:04:49.693425 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 01:04:49.693425 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:04:49.693425 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 30 01:04:49.693425 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 30 01:04:49.693425 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 30 01:04:49.693425 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 30 01:04:49.693425 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 30 01:04:49.693425 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 30 01:04:49.693425 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 30 01:04:49.693425 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 30 01:04:49.693425 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 could not be found. Oct 30 01:04:49.693425 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:04:49.693425 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:04:49.693425 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:04:49.693425 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:04:49.693425 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:04:49.693425 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:04:49.693425 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 01:04:49.693425 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 01:04:49.693425 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 01:04:49.694949 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 01:04:49.694949 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 30 01:04:49.694949 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 30 01:04:49.694949 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 30 01:04:49.694949 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 30 01:04:49.694949 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 30 01:04:49.694949 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 30 01:04:49.694949 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 could not be found. Oct 30 01:04:49.694949 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:04:49.694949 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:04:49.694949 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:04:49.694949 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:04:49.694949 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:04:49.694949 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:04:49.694949 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 01:04:49.694949 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 01:04:49.694949 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 could not be found. Oct 30 01:04:49.694949 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:04:49.696411 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-b21c6d09-7458-4459-a64e-0d9abacc2eeb tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Returning 404 to user: Instance 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 could not be found. {{(pid=118962) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 01:04:49.696411 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-b21c6d09-7458-4459-a64e-0d9abacc2eeb tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 404 len: 111 microversion: 2.1 time: 0.033529 Oct 30 01:04:49.696411 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 496/992] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:04:49 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 30 01:04:49.701450 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-0ec6a9b9-1743-4817-92d4-10fed3e76347 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '.version_select at 0x7f0eaba21e50>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:49.724013 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-0ec6a9b9-1743-4817-92d4-10fed3e76347 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1492790923" status: 202 len: 0 microversion: 2.1 time: 0.026356 Oct 30 01:04:49.724565 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 497/993] 10.222.0.16 () {58 vars in 1221 bytes} [Sun Oct 30 01:04:49 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1492790923 => generated 0 bytes in 28 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 30 01:04:50.381332 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-8191af7b-9a6f-49b6-b637-820be626dc48 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Action: 'create', calling method: .version_select at 0x7f0eafc03820>, body: {"keypair": {"name": "tempest-keypair-1936244065", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPSNbt7QlEfntAk3R3GhHH+Pi1sX3mU4n3IAXVBbmRnV8zHsPMaFZPBG7qBL67wz406kK/DJ9fLx//va4c9mISD5oa1rOzA1dvMb6Nzwmia7Q/3JWivxUzqStcaZwl3Hag=="}} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:04:50.397167 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.quota [None req-8191af7b-9a6f-49b6-b637-820be626dc48 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Getting quotas for project 8bf407954ed34ebfaf56db206df4fc84. Resources: {'key_pairs'} {{(pid=118962) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 30 01:04:50.401188 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.quota [None req-8191af7b-9a6f-49b6-b637-820be626dc48 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Getting quotas for user 1c0e5bd11ac14259ad376ce54bcb7025 and project 8bf407954ed34ebfaf56db206df4fc84. Resources: {'key_pairs'} {{(pid=118962) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 30 01:04:50.425246 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-8191af7b-9a6f-49b6-b637-820be626dc48 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.048014 Oct 30 01:04:50.425832 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 497/994] 10.222.0.16 () {60 vars in 1159 bytes} [Sun Oct 30 01:04:50 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:04:51.762205 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-aa569972-1d1e-43e3-9c9a-d2e920fb7e68 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-2087092903", "imageRef": "4c16f4f6-84cb-4463-b82c-b7c65f3826f0", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "ccfb740b-9abc-42f4-8eda-aa2c80b4bdb0"}], "security_groups": [{"name": "tempest-securitygroup--138214314"}], "key_name": "tempest-keypair-1936244065", "user_data": "}} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:04:51.870221 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.network.neutron [None req-aa569972-1d1e-43e3-9c9a-d2e920fb7e68 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] validate_networks() for [('ccfb740b-9abc-42f4-8eda-aa2c80b4bdb0', None, None, None, None, None)] {{(pid=118961) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 30 01:04:52.254093 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-aa569972-1d1e-43e3-9c9a-d2e920fb7e68 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Flavor limits 0:0:0 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 30 01:04:52.254600 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-aa569972-1d1e-43e3-9c9a-d2e920fb7e68 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Image limits 0:0:0 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 30 01:04:52.255462 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-aa569972-1d1e-43e3-9c9a-d2e920fb7e68 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Flavor pref 0:0:0 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 30 01:04:52.256291 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-aa569972-1d1e-43e3-9c9a-d2e920fb7e68 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Image pref 0:0:0 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 30 01:04:52.256849 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-aa569972-1d1e-43e3-9c9a-d2e920fb7e68 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 30 01:04:52.270035 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-aa569972-1d1e-43e3-9c9a-d2e920fb7e68 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:52.270518 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-aa569972-1d1e-43e3-9c9a-d2e920fb7e68 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:52.270950 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-aa569972-1d1e-43e3-9c9a-d2e920fb7e68 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:52.271674 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-aa569972-1d1e-43e3-9c9a-d2e920fb7e68 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:52.272143 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-aa569972-1d1e-43e3-9c9a-d2e920fb7e68 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:52.272563 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-aa569972-1d1e-43e3-9c9a-d2e920fb7e68 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:52.291082 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.quota [None req-aa569972-1d1e-43e3-9c9a-d2e920fb7e68 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Getting quotas for project 8bf407954ed34ebfaf56db206df4fc84. Resources: {'ram', 'instances', 'cores'} {{(pid=118961) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 30 01:04:52.295463 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.quota [None req-aa569972-1d1e-43e3-9c9a-d2e920fb7e68 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Getting quotas for user 1c0e5bd11ac14259ad376ce54bcb7025 and project 8bf407954ed34ebfaf56db206df4fc84. Resources: {'ram', 'instances', 'cores'} {{(pid=118961) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 30 01:04:52.307946 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-aa569972-1d1e-43e3-9c9a-d2e920fb7e68 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Going to run 1 instances... {{(pid=118961) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 30 01:04:52.310254 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-aa569972-1d1e-43e3-9c9a-d2e920fb7e68 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: d95ac907-83b9-4fe2-b49a-defbe7240d3a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4c16f4f6-84cb-4463-b82c-b7c65f3826f0',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118961) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 30 01:04:52.363291 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-aa569972-1d1e-43e3-9c9a-d2e920fb7e68 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.605997 Oct 30 01:04:52.364580 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 498/995] 10.222.0.16 () {60 vars in 1147 bytes} [Sun Oct 30 01:04:51 2022] POST /compute/v2.1/servers => generated 401 bytes in 608 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 30 01:04:52.375857 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-b3f2586d-b5bc-4f62-8272-3796d7e435ba tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:52.376464 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-b3f2586d-b5bc-4f62-8272-3796d7e435ba tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: d95ac907-83b9-4fe2-b49a-defbe7240d3a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:52.432424 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-b3f2586d-b5bc-4f62-8272-3796d7e435ba tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:52.439193 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-b3f2586d-b5bc-4f62-8272-3796d7e435ba tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a" status: 200 len: 1238 microversion: 2.1 time: 0.067729 Oct 30 01:04:52.439740 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 498/996] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:04:52 2022] GET /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a => generated 1238 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:04:53.463305 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-11717dfd-f377-4e8d-8aff-489f9b944863 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:53.463946 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-11717dfd-f377-4e8d-8aff-489f9b944863 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: d95ac907-83b9-4fe2-b49a-defbe7240d3a] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:53.471642 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-11717dfd-f377-4e8d-8aff-489f9b944863 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:53.472164 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-11717dfd-f377-4e8d-8aff-489f9b944863 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:53.472599 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-11717dfd-f377-4e8d-8aff-489f9b944863 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:53.473438 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-11717dfd-f377-4e8d-8aff-489f9b944863 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:53.473811 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-11717dfd-f377-4e8d-8aff-489f9b944863 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:53.474221 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-11717dfd-f377-4e8d-8aff-489f9b944863 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:53.541763 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-11717dfd-f377-4e8d-8aff-489f9b944863 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:53.550134 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-11717dfd-f377-4e8d-8aff-489f9b944863 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a" status: 200 len: 1290 microversion: 2.1 time: 0.091160 Oct 30 01:04:53.550759 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 499/997] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:04:53 2022] GET /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a => generated 1290 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:04:54.573074 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-75371983-a129-4c57-bcec-0ed850dc7fdf tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:54.573653 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-75371983-a129-4c57-bcec-0ed850dc7fdf tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: d95ac907-83b9-4fe2-b49a-defbe7240d3a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:54.583641 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-75371983-a129-4c57-bcec-0ed850dc7fdf tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:54.584160 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-75371983-a129-4c57-bcec-0ed850dc7fdf tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:54.584591 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-75371983-a129-4c57-bcec-0ed850dc7fdf tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:54.585333 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-75371983-a129-4c57-bcec-0ed850dc7fdf tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:54.585798 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-75371983-a129-4c57-bcec-0ed850dc7fdf tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:54.586545 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-75371983-a129-4c57-bcec-0ed850dc7fdf tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:54.734088 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-75371983-a129-4c57-bcec-0ed850dc7fdf tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:54.742263 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-75371983-a129-4c57-bcec-0ed850dc7fdf tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a" status: 200 len: 1363 microversion: 2.1 time: 0.173204 Oct 30 01:04:54.742855 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 499/998] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:04:54 2022] GET /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a => generated 1363 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:04:55.403747 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [req-0321622c-fa71-452b-8405-92bdbe5039a5 req-2648d87f-3435-4221-bd85-d8a6409c9cf8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d95ac907-83b9-4fe2-b49a-defbe7240d3a", "tag": "b0c592fa-975d-4695-84db-c884d520eff2"}]} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:04:55.414180 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-0321622c-fa71-452b-8405-92bdbe5039a5 req-2648d87f-3435-4221-bd85-d8a6409c9cf8 service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:55.414754 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-0321622c-fa71-452b-8405-92bdbe5039a5 req-2648d87f-3435-4221-bd85-d8a6409c9cf8 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:55.415350 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-0321622c-fa71-452b-8405-92bdbe5039a5 req-2648d87f-3435-4221-bd85-d8a6409c9cf8 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:55.429255 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.compute.server_external_events [req-0321622c-fa71-452b-8405-92bdbe5039a5 req-2648d87f-3435-4221-bd85-d8a6409c9cf8 service nova] Creating event network-changed:b0c592fa-975d-4695-84db-c884d520eff2 for instance d95ac907-83b9-4fe2-b49a-defbe7240d3a on hv-cinder-maste Oct 30 01:04:55.434356 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [req-0321622c-fa71-452b-8405-92bdbe5039a5 req-2648d87f-3435-4221-bd85-d8a6409c9cf8 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034780 Oct 30 01:04:55.434962 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 500/999] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:04:55 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:04:55.766226 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-9ed8f2a4-d3b7-47e9-ae21-6140c914a36f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:55.766943 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-9ed8f2a4-d3b7-47e9-ae21-6140c914a36f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: d95ac907-83b9-4fe2-b49a-defbe7240d3a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:55.773886 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9ed8f2a4-d3b7-47e9-ae21-6140c914a36f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:55.774481 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9ed8f2a4-d3b7-47e9-ae21-6140c914a36f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:55.774907 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9ed8f2a4-d3b7-47e9-ae21-6140c914a36f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:55.775601 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9ed8f2a4-d3b7-47e9-ae21-6140c914a36f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:55.776250 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9ed8f2a4-d3b7-47e9-ae21-6140c914a36f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:55.776672 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9ed8f2a4-d3b7-47e9-ae21-6140c914a36f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:55.916183 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-9ed8f2a4-d3b7-47e9-ae21-6140c914a36f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:55.924646 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-9ed8f2a4-d3b7-47e9-ae21-6140c914a36f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a" status: 200 len: 1363 microversion: 2.1 time: 0.162467 Oct 30 01:04:55.925204 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 500/1000] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:04:55 2022] GET /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a => generated 1363 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:04:56.951387 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-441ebc12-b7f6-461e-a5b6-4ce651d631c1 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:56.952132 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-441ebc12-b7f6-461e-a5b6-4ce651d631c1 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: d95ac907-83b9-4fe2-b49a-defbe7240d3a] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:56.958957 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-441ebc12-b7f6-461e-a5b6-4ce651d631c1 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:56.959438 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-441ebc12-b7f6-461e-a5b6-4ce651d631c1 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:56.959909 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-441ebc12-b7f6-461e-a5b6-4ce651d631c1 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:56.960601 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-441ebc12-b7f6-461e-a5b6-4ce651d631c1 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:56.961431 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-441ebc12-b7f6-461e-a5b6-4ce651d631c1 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:56.961893 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-441ebc12-b7f6-461e-a5b6-4ce651d631c1 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:57.099899 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-441ebc12-b7f6-461e-a5b6-4ce651d631c1 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:57.108251 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-441ebc12-b7f6-461e-a5b6-4ce651d631c1 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a" status: 200 len: 1363 microversion: 2.1 time: 0.162897 Oct 30 01:04:57.108799 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 501/1001] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:04:56 2022] GET /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a => generated 1363 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:04:58.130954 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-c33bc116-8cb3-42dc-ad61-07a4db43ab4d tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:58.131486 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-c33bc116-8cb3-42dc-ad61-07a4db43ab4d tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: d95ac907-83b9-4fe2-b49a-defbe7240d3a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:58.138536 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c33bc116-8cb3-42dc-ad61-07a4db43ab4d tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:58.139021 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c33bc116-8cb3-42dc-ad61-07a4db43ab4d tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:58.139451 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c33bc116-8cb3-42dc-ad61-07a4db43ab4d tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:58.140177 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c33bc116-8cb3-42dc-ad61-07a4db43ab4d tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:58.140626 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c33bc116-8cb3-42dc-ad61-07a4db43ab4d tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:58.141046 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c33bc116-8cb3-42dc-ad61-07a4db43ab4d tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:58.274047 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-c33bc116-8cb3-42dc-ad61-07a4db43ab4d tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:58.282635 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-c33bc116-8cb3-42dc-ad61-07a4db43ab4d tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a" status: 200 len: 1363 microversion: 2.1 time: 0.155805 Oct 30 01:04:58.283201 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 501/1002] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:04:58 2022] GET /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a => generated 1363 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:04:59.314829 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-ce9ecf17-1ed2-437e-9503-a3b863c6c865 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:04:59.315538 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-ce9ecf17-1ed2-437e-9503-a3b863c6c865 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: d95ac907-83b9-4fe2-b49a-defbe7240d3a] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:04:59.322947 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ce9ecf17-1ed2-437e-9503-a3b863c6c865 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:59.323425 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ce9ecf17-1ed2-437e-9503-a3b863c6c865 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:59.323887 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ce9ecf17-1ed2-437e-9503-a3b863c6c865 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:59.324579 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ce9ecf17-1ed2-437e-9503-a3b863c6c865 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:04:59.325026 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ce9ecf17-1ed2-437e-9503-a3b863c6c865 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:04:59.325535 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ce9ecf17-1ed2-437e-9503-a3b863c6c865 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:04:59.457207 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-ce9ecf17-1ed2-437e-9503-a3b863c6c865 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:04:59.465821 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-ce9ecf17-1ed2-437e-9503-a3b863c6c865 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a" status: 200 len: 1363 microversion: 2.1 time: 0.155043 Oct 30 01:04:59.466431 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 502/1003] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:04:59 2022] GET /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a => generated 1363 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:05:00.489091 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-aa50d2c7-0bc3-426e-8000-8a0488a1fc47 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:05:00.489827 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-aa50d2c7-0bc3-426e-8000-8a0488a1fc47 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: d95ac907-83b9-4fe2-b49a-defbe7240d3a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:05:00.496656 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-aa50d2c7-0bc3-426e-8000-8a0488a1fc47 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:00.497175 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-aa50d2c7-0bc3-426e-8000-8a0488a1fc47 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:00.497608 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-aa50d2c7-0bc3-426e-8000-8a0488a1fc47 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:00.498309 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-aa50d2c7-0bc3-426e-8000-8a0488a1fc47 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:00.498754 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-aa50d2c7-0bc3-426e-8000-8a0488a1fc47 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:00.499173 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-aa50d2c7-0bc3-426e-8000-8a0488a1fc47 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:00.633574 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-aa50d2c7-0bc3-426e-8000-8a0488a1fc47 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:05:00.641682 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-aa50d2c7-0bc3-426e-8000-8a0488a1fc47 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a" status: 200 len: 1363 microversion: 2.1 time: 0.156661 Oct 30 01:05:00.642273 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 502/1004] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:05:00 2022] GET /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a => generated 1363 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:05:01.666600 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-f1a17f7b-0a3e-427a-ad46-aa372699f4bb tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:05:01.667397 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-f1a17f7b-0a3e-427a-ad46-aa372699f4bb tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: d95ac907-83b9-4fe2-b49a-defbe7240d3a] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:05:01.674319 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f1a17f7b-0a3e-427a-ad46-aa372699f4bb tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:01.674807 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f1a17f7b-0a3e-427a-ad46-aa372699f4bb tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:01.675239 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f1a17f7b-0a3e-427a-ad46-aa372699f4bb tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:01.675976 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f1a17f7b-0a3e-427a-ad46-aa372699f4bb tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:01.676422 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f1a17f7b-0a3e-427a-ad46-aa372699f4bb tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:01.676844 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f1a17f7b-0a3e-427a-ad46-aa372699f4bb tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:01.810604 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-f1a17f7b-0a3e-427a-ad46-aa372699f4bb tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:05:01.818804 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-f1a17f7b-0a3e-427a-ad46-aa372699f4bb tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a" status: 200 len: 1363 microversion: 2.1 time: 0.157416 Oct 30 01:05:01.819380 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 503/1005] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:05:01 2022] GET /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a => generated 1363 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:05:02.000644 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [req-e231decb-8325-4ef4-afc3-ed257be1e2f9 req-be1169a0-eed3-4cd3-bdf4-a9693507136c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d95ac907-83b9-4fe2-b49a-defbe7240d3a", "name": "network-vif-plugged", "status": "completed", "tag": "b0c592fa-975d-4695-84db-c884d520eff2"}]} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:05:02.011159 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-e231decb-8325-4ef4-afc3-ed257be1e2f9 req-be1169a0-eed3-4cd3-bdf4-a9693507136c service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:02.011656 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-e231decb-8325-4ef4-afc3-ed257be1e2f9 req-be1169a0-eed3-4cd3-bdf4-a9693507136c service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:02.012231 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-e231decb-8325-4ef4-afc3-ed257be1e2f9 req-be1169a0-eed3-4cd3-bdf4-a9693507136c service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:02.023991 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.compute.server_external_events [req-e231decb-8325-4ef4-afc3-ed257be1e2f9 req-be1169a0-eed3-4cd3-bdf4-a9693507136c service nova] Creating event network-vif-plugged:b0c592fa-975d-4695-84db-c884d520eff2 for instance d95ac907-83b9-4fe2-b49a-defbe7240d3a on hv-cinder-maste Oct 30 01:05:02.029161 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [req-e231decb-8325-4ef4-afc3-ed257be1e2f9 req-be1169a0-eed3-4cd3-bdf4-a9693507136c service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032379 Oct 30 01:05:02.029752 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 503/1006] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:05:01 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:05:02.846187 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-28c53d76-1e47-4bd8-b3e8-1a2e540837b2 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:05:02.847196 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-28c53d76-1e47-4bd8-b3e8-1a2e540837b2 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: d95ac907-83b9-4fe2-b49a-defbe7240d3a] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:05:02.850280 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell0:SELECT=2 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:05:02.851455 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell0:SELECT=6 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:05:02.859846 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-28c53d76-1e47-4bd8-b3e8-1a2e540837b2 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:02.860325 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-28c53d76-1e47-4bd8-b3e8-1a2e540837b2 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:02.860756 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-28c53d76-1e47-4bd8-b3e8-1a2e540837b2 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:02.861447 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-28c53d76-1e47-4bd8-b3e8-1a2e540837b2 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:02.861897 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-28c53d76-1e47-4bd8-b3e8-1a2e540837b2 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:02.862314 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-28c53d76-1e47-4bd8-b3e8-1a2e540837b2 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:03.005534 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-28c53d76-1e47-4bd8-b3e8-1a2e540837b2 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:05:03.013630 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-28c53d76-1e47-4bd8-b3e8-1a2e540837b2 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a" status: 200 len: 1363 microversion: 2.1 time: 0.174034 Oct 30 01:05:03.014172 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 504/1007] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:05:02 2022] GET /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a => generated 1363 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:05:04.034299 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell0:SELECT=6 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:05:04.039466 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-02d20c24-9753-4062-8333-a010e0aaa057 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:05:04.040376 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-02d20c24-9753-4062-8333-a010e0aaa057 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: d95ac907-83b9-4fe2-b49a-defbe7240d3a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:05:04.047166 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-02d20c24-9753-4062-8333-a010e0aaa057 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:04.047708 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-02d20c24-9753-4062-8333-a010e0aaa057 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:04.048128 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-02d20c24-9753-4062-8333-a010e0aaa057 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:04.048809 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-02d20c24-9753-4062-8333-a010e0aaa057 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:04.049252 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-02d20c24-9753-4062-8333-a010e0aaa057 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:04.049670 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-02d20c24-9753-4062-8333-a010e0aaa057 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:04.184642 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-02d20c24-9753-4062-8333-a010e0aaa057 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:05:04.192614 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-02d20c24-9753-4062-8333-a010e0aaa057 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a" status: 200 len: 1550 microversion: 2.1 time: 0.159332 Oct 30 01:05:04.193175 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 504/1008] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:05:04 2022] GET /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a => generated 1550 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:05:04.223485 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-a5f02745-cb82-4ec5-9f86-629bd0d22eb4 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:05:04.224039 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-a5f02745-cb82-4ec5-9f86-629bd0d22eb4 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: d95ac907-83b9-4fe2-b49a-defbe7240d3a] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:05:04.230951 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a5f02745-cb82-4ec5-9f86-629bd0d22eb4 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:04.231432 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a5f02745-cb82-4ec5-9f86-629bd0d22eb4 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:04.231904 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a5f02745-cb82-4ec5-9f86-629bd0d22eb4 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:04.232692 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a5f02745-cb82-4ec5-9f86-629bd0d22eb4 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:04.233140 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a5f02745-cb82-4ec5-9f86-629bd0d22eb4 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:04.233582 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a5f02745-cb82-4ec5-9f86-629bd0d22eb4 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:04.310437 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-a5f02745-cb82-4ec5-9f86-629bd0d22eb4 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.090860 Oct 30 01:05:04.311068 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 505/1009] 10.222.0.16 () {58 vars in 1275 bytes} [Sun Oct 30 01:05:04 2022] GET /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a/os-interface => generated 285 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:05:05.906148 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [req-59a77dd3-4f34-400e-88d5-5b0af180510a req-f0038950-e0a4-4ec3-a7a5-c86523757557 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d95ac907-83b9-4fe2-b49a-defbe7240d3a", "tag": "b0c592fa-975d-4695-84db-c884d520eff2"}]} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:05:05.916294 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-59a77dd3-4f34-400e-88d5-5b0af180510a req-f0038950-e0a4-4ec3-a7a5-c86523757557 service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:05.916839 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-59a77dd3-4f34-400e-88d5-5b0af180510a req-f0038950-e0a4-4ec3-a7a5-c86523757557 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:05.917326 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-59a77dd3-4f34-400e-88d5-5b0af180510a req-f0038950-e0a4-4ec3-a7a5-c86523757557 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:05.928486 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.compute.server_external_events [req-59a77dd3-4f34-400e-88d5-5b0af180510a req-f0038950-e0a4-4ec3-a7a5-c86523757557 service nova] Creating event network-changed:b0c592fa-975d-4695-84db-c884d520eff2 for instance d95ac907-83b9-4fe2-b49a-defbe7240d3a on hv-cinder-maste Oct 30 01:05:05.933605 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [req-59a77dd3-4f34-400e-88d5-5b0af180510a req-f0038950-e0a4-4ec3-a7a5-c86523757557 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031235 Oct 30 01:05:05.934437 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 505/1010] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:05:05 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:05:09.036570 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-6eae96a8-bc04-43ca-946f-7d480c07d4bf tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Action: 'action', calling method: >, body: {"reboot": {"type": "SOFT"}} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:05:09.039537 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-6eae96a8-bc04-43ca-946f-7d480c07d4bf tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:05:09.047601 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6eae96a8-bc04-43ca-946f-7d480c07d4bf tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:09.048122 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6eae96a8-bc04-43ca-946f-7d480c07d4bf tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:09.048554 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6eae96a8-bc04-43ca-946f-7d480c07d4bf tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:09.049240 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6eae96a8-bc04-43ca-946f-7d480c07d4bf tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:09.049690 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6eae96a8-bc04-43ca-946f-7d480c07d4bf tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:09.050112 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6eae96a8-bc04-43ca-946f-7d480c07d4bf tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:09.130137 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-6eae96a8-bc04-43ca-946f-7d480c07d4bf tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "POST /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/action" status: 202 len: 0 microversion: 2.1 time: 0.099668 Oct 30 01:05:09.131038 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 506/1011] 10.222.0.16 () {60 vars in 1278 bytes} [Sun Oct 30 01:05:09 2022] POST /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/action => generated 0 bytes in 101 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 30 01:05:09.140188 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-e5d3ca84-c5bc-444d-98ce-8a8c2d8a67bc tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:05:09.140888 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-e5d3ca84-c5bc-444d-98ce-8a8c2d8a67bc tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:05:09.146963 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e5d3ca84-c5bc-444d-98ce-8a8c2d8a67bc tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:09.147472 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e5d3ca84-c5bc-444d-98ce-8a8c2d8a67bc tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:09.147925 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e5d3ca84-c5bc-444d-98ce-8a8c2d8a67bc tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:09.148623 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e5d3ca84-c5bc-444d-98ce-8a8c2d8a67bc tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:09.149076 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e5d3ca84-c5bc-444d-98ce-8a8c2d8a67bc tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:09.149463 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e5d3ca84-c5bc-444d-98ce-8a8c2d8a67bc tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:09.297093 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-e5d3ca84-c5bc-444d-98ce-8a8c2d8a67bc tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '05eed70effab4e38ae469ab44d1114db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dcbf2e9fb2e4a4698633fcbb07c30cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:05:09.305462 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-e5d3ca84-c5bc-444d-98ce-8a8c2d8a67bc tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7" status: 200 len: 1752 microversion: 2.1 time: 0.170016 Oct 30 01:05:09.305958 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 506/1012] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:05:09 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7 => generated 1752 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:05:10.331462 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-0b40bada-9a26-401a-a667-2f57fa8a0a56 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:05:10.332055 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-0b40bada-9a26-401a-a667-2f57fa8a0a56 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:05:10.339502 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0b40bada-9a26-401a-a667-2f57fa8a0a56 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:10.340019 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0b40bada-9a26-401a-a667-2f57fa8a0a56 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:10.340450 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0b40bada-9a26-401a-a667-2f57fa8a0a56 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:10.341179 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0b40bada-9a26-401a-a667-2f57fa8a0a56 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:10.341602 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0b40bada-9a26-401a-a667-2f57fa8a0a56 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:10.342046 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0b40bada-9a26-401a-a667-2f57fa8a0a56 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:10.491927 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-0b40bada-9a26-401a-a667-2f57fa8a0a56 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '05eed70effab4e38ae469ab44d1114db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dcbf2e9fb2e4a4698633fcbb07c30cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:05:10.505380 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-0b40bada-9a26-401a-a667-2f57fa8a0a56 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7" status: 200 len: 1757 microversion: 2.1 time: 0.178038 Oct 30 01:05:10.506260 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 507/1013] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:05:10 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7 => generated 1757 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:05:11.533865 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-31a8388f-33e2-468a-8d56-4fe0399df934 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:05:11.534907 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-31a8388f-33e2-468a-8d56-4fe0399df934 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:05:11.546199 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-31a8388f-33e2-468a-8d56-4fe0399df934 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:11.546980 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-31a8388f-33e2-468a-8d56-4fe0399df934 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:11.547408 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-31a8388f-33e2-468a-8d56-4fe0399df934 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:11.548135 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-31a8388f-33e2-468a-8d56-4fe0399df934 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:11.548596 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-31a8388f-33e2-468a-8d56-4fe0399df934 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:11.549012 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-31a8388f-33e2-468a-8d56-4fe0399df934 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:11.939066 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-31a8388f-33e2-468a-8d56-4fe0399df934 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '05eed70effab4e38ae469ab44d1114db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dcbf2e9fb2e4a4698633fcbb07c30cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:05:11.950170 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-31a8388f-33e2-468a-8d56-4fe0399df934 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7" status: 200 len: 1757 microversion: 2.1 time: 0.422765 Oct 30 01:05:11.950961 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 507/1014] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:05:11 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7 => generated 1757 bytes in 425 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:05:12.985206 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-06ac3d42-85f7-46d0-a939-2bcab82bcdea tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:05:12.985764 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-06ac3d42-85f7-46d0-a939-2bcab82bcdea tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:05:12.992828 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-06ac3d42-85f7-46d0-a939-2bcab82bcdea tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:12.993306 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-06ac3d42-85f7-46d0-a939-2bcab82bcdea tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:12.993735 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-06ac3d42-85f7-46d0-a939-2bcab82bcdea tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:12.994421 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-06ac3d42-85f7-46d0-a939-2bcab82bcdea tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:12.994860 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-06ac3d42-85f7-46d0-a939-2bcab82bcdea tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:12.995278 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-06ac3d42-85f7-46d0-a939-2bcab82bcdea tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:13.157455 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-06ac3d42-85f7-46d0-a939-2bcab82bcdea tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '05eed70effab4e38ae469ab44d1114db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dcbf2e9fb2e4a4698633fcbb07c30cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:05:13.170434 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-06ac3d42-85f7-46d0-a939-2bcab82bcdea tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7" status: 200 len: 1757 microversion: 2.1 time: 0.191319 Oct 30 01:05:13.171289 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 508/1015] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:05:12 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7 => generated 1757 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:05:14.206866 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-b1237be9-c490-40f7-83f4-7f2fc4dfa9ea tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:05:14.207424 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-b1237be9-c490-40f7-83f4-7f2fc4dfa9ea tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:05:14.214775 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b1237be9-c490-40f7-83f4-7f2fc4dfa9ea tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:14.215259 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b1237be9-c490-40f7-83f4-7f2fc4dfa9ea tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:14.215688 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b1237be9-c490-40f7-83f4-7f2fc4dfa9ea tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:14.216410 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b1237be9-c490-40f7-83f4-7f2fc4dfa9ea tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:14.216860 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b1237be9-c490-40f7-83f4-7f2fc4dfa9ea tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:14.217278 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b1237be9-c490-40f7-83f4-7f2fc4dfa9ea tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:14.363921 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-b1237be9-c490-40f7-83f4-7f2fc4dfa9ea tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '05eed70effab4e38ae469ab44d1114db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dcbf2e9fb2e4a4698633fcbb07c30cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:05:14.372426 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-b1237be9-c490-40f7-83f4-7f2fc4dfa9ea tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7" status: 200 len: 1757 microversion: 2.1 time: 0.170298 Oct 30 01:05:14.373026 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 508/1016] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:05:14 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7 => generated 1757 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:05:15.397073 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-f483f8f6-5e97-443b-ac89-6a93310a7a4d tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:05:15.397634 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-f483f8f6-5e97-443b-ac89-6a93310a7a4d tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:05:15.400231 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_api:SELECT=49,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:05:15.405698 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f483f8f6-5e97-443b-ac89-6a93310a7a4d tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:15.406259 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f483f8f6-5e97-443b-ac89-6a93310a7a4d tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:15.406722 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f483f8f6-5e97-443b-ac89-6a93310a7a4d tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:15.407478 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f483f8f6-5e97-443b-ac89-6a93310a7a4d tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:15.407969 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f483f8f6-5e97-443b-ac89-6a93310a7a4d tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:15.408394 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f483f8f6-5e97-443b-ac89-6a93310a7a4d tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:15.411742 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell1:SELECT=87,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:05:15.557145 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-f483f8f6-5e97-443b-ac89-6a93310a7a4d tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '05eed70effab4e38ae469ab44d1114db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dcbf2e9fb2e4a4698633fcbb07c30cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:05:15.565736 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-f483f8f6-5e97-443b-ac89-6a93310a7a4d tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7" status: 200 len: 1757 microversion: 2.1 time: 0.172619 Oct 30 01:05:15.566277 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 509/1017] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:05:15 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7 => generated 1757 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:05:16.590146 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-5e0524ae-8a82-45c3-93bc-a299e2fc559a tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:05:16.590759 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-5e0524ae-8a82-45c3-93bc-a299e2fc559a tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:05:16.593301 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_api:SELECT=49,nova_api:UPDATE=1,nova_api:INSERT=1 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:05:16.598934 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5e0524ae-8a82-45c3-93bc-a299e2fc559a tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:16.599408 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5e0524ae-8a82-45c3-93bc-a299e2fc559a tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:16.599892 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5e0524ae-8a82-45c3-93bc-a299e2fc559a tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:16.600583 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5e0524ae-8a82-45c3-93bc-a299e2fc559a tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:16.601023 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5e0524ae-8a82-45c3-93bc-a299e2fc559a tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:16.601441 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5e0524ae-8a82-45c3-93bc-a299e2fc559a tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:16.604967 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell1:SELECT=77,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:05:16.748302 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-5e0524ae-8a82-45c3-93bc-a299e2fc559a tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '05eed70effab4e38ae469ab44d1114db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dcbf2e9fb2e4a4698633fcbb07c30cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:05:16.760339 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-5e0524ae-8a82-45c3-93bc-a299e2fc559a tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7" status: 200 len: 1757 microversion: 2.1 time: 0.174186 Oct 30 01:05:16.761142 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 509/1018] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:05:16 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7 => generated 1757 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:05:17.792604 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-d73c1b98-9e0e-4212-b767-d140308b6e26 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:05:17.793218 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-d73c1b98-9e0e-4212-b767-d140308b6e26 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:05:17.800223 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d73c1b98-9e0e-4212-b767-d140308b6e26 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:17.800708 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d73c1b98-9e0e-4212-b767-d140308b6e26 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:17.801140 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d73c1b98-9e0e-4212-b767-d140308b6e26 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:17.801833 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d73c1b98-9e0e-4212-b767-d140308b6e26 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:17.802276 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d73c1b98-9e0e-4212-b767-d140308b6e26 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:17.802692 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d73c1b98-9e0e-4212-b767-d140308b6e26 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:17.956695 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-d73c1b98-9e0e-4212-b767-d140308b6e26 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '05eed70effab4e38ae469ab44d1114db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dcbf2e9fb2e4a4698633fcbb07c30cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:05:17.969422 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-d73c1b98-9e0e-4212-b767-d140308b6e26 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7" status: 200 len: 1757 microversion: 2.1 time: 0.180634 Oct 30 01:05:17.970064 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 510/1019] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:05:17 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7 => generated 1757 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:05:19.001939 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-3ac5d152-a78e-4b2f-adf1-86909843bf4e tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:05:19.002547 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-3ac5d152-a78e-4b2f-adf1-86909843bf4e tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:05:19.009372 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3ac5d152-a78e-4b2f-adf1-86909843bf4e tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:19.009852 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3ac5d152-a78e-4b2f-adf1-86909843bf4e tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:19.010275 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3ac5d152-a78e-4b2f-adf1-86909843bf4e tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:19.010956 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3ac5d152-a78e-4b2f-adf1-86909843bf4e tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:19.011395 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3ac5d152-a78e-4b2f-adf1-86909843bf4e tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:19.011838 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3ac5d152-a78e-4b2f-adf1-86909843bf4e tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:19.157839 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-3ac5d152-a78e-4b2f-adf1-86909843bf4e tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '05eed70effab4e38ae469ab44d1114db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dcbf2e9fb2e4a4698633fcbb07c30cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:05:19.166082 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-3ac5d152-a78e-4b2f-adf1-86909843bf4e tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7" status: 200 len: 1757 microversion: 2.1 time: 0.170053 Oct 30 01:05:19.166649 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 510/1020] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:05:18 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7 => generated 1757 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:05:20.193250 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-89711bd7-c223-4383-b7b4-52cd8a3de03f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:05:20.194015 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-89711bd7-c223-4383-b7b4-52cd8a3de03f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:05:20.204107 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-89711bd7-c223-4383-b7b4-52cd8a3de03f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:20.204845 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-89711bd7-c223-4383-b7b4-52cd8a3de03f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:20.205613 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-89711bd7-c223-4383-b7b4-52cd8a3de03f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:20.206396 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-89711bd7-c223-4383-b7b4-52cd8a3de03f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:20.206845 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-89711bd7-c223-4383-b7b4-52cd8a3de03f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:20.207263 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-89711bd7-c223-4383-b7b4-52cd8a3de03f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:20.355227 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-89711bd7-c223-4383-b7b4-52cd8a3de03f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '05eed70effab4e38ae469ab44d1114db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dcbf2e9fb2e4a4698633fcbb07c30cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:05:20.364332 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-89711bd7-c223-4383-b7b4-52cd8a3de03f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7" status: 200 len: 1757 microversion: 2.1 time: 0.177525 Oct 30 01:05:20.364964 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 511/1021] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:05:20 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7 => generated 1757 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:05:21.389380 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-5236c7d9-0c3f-4f26-b6a8-d88a0120a02f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:05:21.390005 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-5236c7d9-0c3f-4f26-b6a8-d88a0120a02f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:05:21.396619 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5236c7d9-0c3f-4f26-b6a8-d88a0120a02f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:21.397102 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5236c7d9-0c3f-4f26-b6a8-d88a0120a02f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:21.397526 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5236c7d9-0c3f-4f26-b6a8-d88a0120a02f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:21.398219 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5236c7d9-0c3f-4f26-b6a8-d88a0120a02f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:21.398689 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5236c7d9-0c3f-4f26-b6a8-d88a0120a02f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:21.399134 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5236c7d9-0c3f-4f26-b6a8-d88a0120a02f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:21.547892 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-5236c7d9-0c3f-4f26-b6a8-d88a0120a02f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '05eed70effab4e38ae469ab44d1114db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dcbf2e9fb2e4a4698633fcbb07c30cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:05:21.556531 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-5236c7d9-0c3f-4f26-b6a8-d88a0120a02f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7" status: 200 len: 1760 microversion: 2.1 time: 0.171881 Oct 30 01:05:21.557105 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 511/1022] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:05:21 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7 => generated 1760 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:05:33.367198 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-93d9df9f-8ec1-47e8-a3c6-1a90fb719cb1 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:05:33.367795 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-93d9df9f-8ec1-47e8-a3c6-1a90fb719cb1 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:05:33.369906 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_api:SELECT=5 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:05:33.370556 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell1:SELECT=17 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:05:33.377381 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-93d9df9f-8ec1-47e8-a3c6-1a90fb719cb1 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:33.377868 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-93d9df9f-8ec1-47e8-a3c6-1a90fb719cb1 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:33.378301 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-93d9df9f-8ec1-47e8-a3c6-1a90fb719cb1 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:33.378982 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-93d9df9f-8ec1-47e8-a3c6-1a90fb719cb1 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:33.379430 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-93d9df9f-8ec1-47e8-a3c6-1a90fb719cb1 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:33.379883 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-93d9df9f-8ec1-47e8-a3c6-1a90fb719cb1 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:33.477295 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.objects.instance [None req-93d9df9f-8ec1-47e8-a3c6-1a90fb719cb1 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lazy-loading 'services' on Instance uuid de41579d-0173-4dba-962d-a28c85c505de {{(pid=118961) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 30 01:05:33.566143 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-93d9df9f-8ec1-47e8-a3c6-1a90fb719cb1 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "DELETE /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de/os-volume_attachments/5eb9e9ce-78a3-49b0-b099-d73664815017" status: 202 len: 0 microversion: 2.1 time: 0.202763 Oct 30 01:05:33.566755 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 512/1023] 10.222.0.16 () {58 vars in 1416 bytes} [Sun Oct 30 01:05:33 2022] DELETE /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de/os-volume_attachments/5eb9e9ce-78a3-49b0-b099-d73664815017 => generated 0 bytes in 204 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 30 01:05:33.577176 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-d85f946b-0146-44de-817f-887bb7da1460 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:05:33.579662 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-d85f946b-0146-44de-817f-887bb7da1460 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:05:33.582295 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_api:SELECT=5 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:05:33.583215 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell1:SELECT=17 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:05:33.589658 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d85f946b-0146-44de-817f-887bb7da1460 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:33.590297 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d85f946b-0146-44de-817f-887bb7da1460 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:33.590791 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d85f946b-0146-44de-817f-887bb7da1460 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:33.591545 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d85f946b-0146-44de-817f-887bb7da1460 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:33.592089 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d85f946b-0146-44de-817f-887bb7da1460 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:33.592504 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d85f946b-0146-44de-817f-887bb7da1460 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:33.624580 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-d85f946b-0146-44de-817f-887bb7da1460 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052565 Oct 30 01:05:33.625145 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 512/1024] 10.222.0.16 () {58 vars in 1302 bytes} [Sun Oct 30 01:05:33 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:05:34.642530 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-43691e60-2baa-41dd-ac6d-01cf480a3930 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:05:34.646333 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-43691e60-2baa-41dd-ac6d-01cf480a3930 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:05:34.658206 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-43691e60-2baa-41dd-ac6d-01cf480a3930 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:34.659054 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-43691e60-2baa-41dd-ac6d-01cf480a3930 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:34.659792 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-43691e60-2baa-41dd-ac6d-01cf480a3930 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:34.660937 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-43691e60-2baa-41dd-ac6d-01cf480a3930 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:34.661679 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-43691e60-2baa-41dd-ac6d-01cf480a3930 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:34.662377 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-43691e60-2baa-41dd-ac6d-01cf480a3930 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:34.693107 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-43691e60-2baa-41dd-ac6d-01cf480a3930 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058017 Oct 30 01:05:34.693656 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 513/1025] 10.222.0.16 () {58 vars in 1302 bytes} [Sun Oct 30 01:05:34 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:05:35.710561 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-3b1c026c-9c9c-4e1b-9de3-155842df0647 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:05:35.713530 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-3b1c026c-9c9c-4e1b-9de3-155842df0647 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:05:35.720752 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3b1c026c-9c9c-4e1b-9de3-155842df0647 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:35.721254 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3b1c026c-9c9c-4e1b-9de3-155842df0647 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:35.721706 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3b1c026c-9c9c-4e1b-9de3-155842df0647 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:35.722440 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3b1c026c-9c9c-4e1b-9de3-155842df0647 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:35.722915 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3b1c026c-9c9c-4e1b-9de3-155842df0647 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:35.723362 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3b1c026c-9c9c-4e1b-9de3-155842df0647 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:35.756336 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-3b1c026c-9c9c-4e1b-9de3-155842df0647 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.052634 Oct 30 01:05:35.756953 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 513/1026] 10.222.0.16 () {58 vars in 1302 bytes} [Sun Oct 30 01:05:35 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de/os-volume_attachments => generated 25 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 30 01:05:35.844825 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-9808ba0e-9bd9-4f3e-bfe1-a9d13fccd87a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:05:35.845428 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-9808ba0e-9bd9-4f3e-bfe1-a9d13fccd87a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:05:35.852437 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9808ba0e-9bd9-4f3e-bfe1-a9d13fccd87a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:35.852919 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9808ba0e-9bd9-4f3e-bfe1-a9d13fccd87a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:35.853355 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9808ba0e-9bd9-4f3e-bfe1-a9d13fccd87a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:35.854056 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9808ba0e-9bd9-4f3e-bfe1-a9d13fccd87a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:35.854511 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9808ba0e-9bd9-4f3e-bfe1-a9d13fccd87a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:35.854917 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9808ba0e-9bd9-4f3e-bfe1-a9d13fccd87a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:35.882303 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-9808ba0e-9bd9-4f3e-bfe1-a9d13fccd87a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Going to try to terminate instance {{(pid=118961) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 30 01:05:35.968787 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-9808ba0e-9bd9-4f3e-bfe1-a9d13fccd87a tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "DELETE /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 204 len: 0 microversion: 2.1 time: 0.127527 Oct 30 01:05:35.969568 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 514/1027] 10.222.0.16 () {58 vars in 1239 bytes} [Sun Oct 30 01:05:35 2022] DELETE /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 0 bytes in 129 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Oct 30 01:05:35.979010 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-60b3e556-36fb-483f-9b46-cf8e7f44fc50 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:05:35.979611 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-60b3e556-36fb-483f-9b46-cf8e7f44fc50 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:05:35.985643 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-60b3e556-36fb-483f-9b46-cf8e7f44fc50 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:35.986121 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-60b3e556-36fb-483f-9b46-cf8e7f44fc50 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:35.986549 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-60b3e556-36fb-483f-9b46-cf8e7f44fc50 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:35.987238 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-60b3e556-36fb-483f-9b46-cf8e7f44fc50 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:35.987686 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-60b3e556-36fb-483f-9b46-cf8e7f44fc50 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:35.988131 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-60b3e556-36fb-483f-9b46-cf8e7f44fc50 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:36.122329 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-60b3e556-36fb-483f-9b46-cf8e7f44fc50 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:05:36.130716 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-60b3e556-36fb-483f-9b46-cf8e7f44fc50 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1690 microversion: 2.1 time: 0.155786 Oct 30 01:05:36.131301 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 514/1028] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:05:35 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1690 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:05:37.159553 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-765982f3-b392-4110-88b7-302acb8410d5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:05:37.160753 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-765982f3-b392-4110-88b7-302acb8410d5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:05:37.172157 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-765982f3-b392-4110-88b7-302acb8410d5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:37.172976 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-765982f3-b392-4110-88b7-302acb8410d5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:37.173710 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-765982f3-b392-4110-88b7-302acb8410d5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:37.174879 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-765982f3-b392-4110-88b7-302acb8410d5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:37.175648 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-765982f3-b392-4110-88b7-302acb8410d5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:37.176417 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-765982f3-b392-4110-88b7-302acb8410d5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:37.330683 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-765982f3-b392-4110-88b7-302acb8410d5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:05:37.339301 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-765982f3-b392-4110-88b7-302acb8410d5 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1690 microversion: 2.1 time: 0.187112 Oct 30 01:05:37.340198 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 515/1029] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:05:37 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1690 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:05:38.363837 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-5c71dd23-afef-4892-9e6b-c81865e4f9b4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:05:38.364442 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-5c71dd23-afef-4892-9e6b-c81865e4f9b4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:05:38.373469 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5c71dd23-afef-4892-9e6b-c81865e4f9b4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:38.374158 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5c71dd23-afef-4892-9e6b-c81865e4f9b4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:38.374695 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5c71dd23-afef-4892-9e6b-c81865e4f9b4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:38.375766 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5c71dd23-afef-4892-9e6b-c81865e4f9b4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:38.376415 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5c71dd23-afef-4892-9e6b-c81865e4f9b4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:38.377008 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5c71dd23-afef-4892-9e6b-c81865e4f9b4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:38.512489 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-5c71dd23-afef-4892-9e6b-c81865e4f9b4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:05:38.520887 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-5c71dd23-afef-4892-9e6b-c81865e4f9b4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1690 microversion: 2.1 time: 0.162276 Oct 30 01:05:38.521459 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 515/1030] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:05:38 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1690 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:05:39.544215 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-b4f16cf7-187f-462a-83cc-0a675d2b46fb tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:05:39.544851 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-b4f16cf7-187f-462a-83cc-0a675d2b46fb tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:05:39.552258 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b4f16cf7-187f-462a-83cc-0a675d2b46fb tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:39.552670 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b4f16cf7-187f-462a-83cc-0a675d2b46fb tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:39.553360 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b4f16cf7-187f-462a-83cc-0a675d2b46fb tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:39.554272 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b4f16cf7-187f-462a-83cc-0a675d2b46fb tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:39.554564 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b4f16cf7-187f-462a-83cc-0a675d2b46fb tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:39.555137 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b4f16cf7-187f-462a-83cc-0a675d2b46fb tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:39.703053 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-b4f16cf7-187f-462a-83cc-0a675d2b46fb tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:05:39.715154 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-b4f16cf7-187f-462a-83cc-0a675d2b46fb tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1690 microversion: 2.1 time: 0.174554 Oct 30 01:05:39.716034 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 516/1031] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:05:39 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1690 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:05:39.803596 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-500aeb0e-15f6-428a-8fdc-4184c518f6d6 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:05:39.804412 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-500aeb0e-15f6-428a-8fdc-4184c518f6d6 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: b77a1841-4f87-4b8a-96a9-a619ea67d452] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:05:39.815060 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-500aeb0e-15f6-428a-8fdc-4184c518f6d6 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:39.815786 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-500aeb0e-15f6-428a-8fdc-4184c518f6d6 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:39.816472 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-500aeb0e-15f6-428a-8fdc-4184c518f6d6 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:39.817525 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-500aeb0e-15f6-428a-8fdc-4184c518f6d6 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:39.818209 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-500aeb0e-15f6-428a-8fdc-4184c518f6d6 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:39.818842 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-500aeb0e-15f6-428a-8fdc-4184c518f6d6 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:39.854112 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-500aeb0e-15f6-428a-8fdc-4184c518f6d6 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452/ips" status: 200 len: 150 microversion: 2.1 time: 0.057808 Oct 30 01:05:39.854688 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 516/1032] 10.222.0.16 () {58 vars in 1248 bytes} [Sun Oct 30 01:05:39 2022] GET /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452/ips => generated 150 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:05:40.742426 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-1ae00aff-a711-47a6-a808-14cb799edb8c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:05:40.742996 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-1ae00aff-a711-47a6-a808-14cb799edb8c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:05:40.750154 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1ae00aff-a711-47a6-a808-14cb799edb8c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:40.750635 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1ae00aff-a711-47a6-a808-14cb799edb8c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:40.751059 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1ae00aff-a711-47a6-a808-14cb799edb8c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:40.751752 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1ae00aff-a711-47a6-a808-14cb799edb8c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:40.752224 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1ae00aff-a711-47a6-a808-14cb799edb8c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:40.752665 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1ae00aff-a711-47a6-a808-14cb799edb8c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:40.890773 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-1ae00aff-a711-47a6-a808-14cb799edb8c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:05:40.899037 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-1ae00aff-a711-47a6-a808-14cb799edb8c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1690 microversion: 2.1 time: 0.160521 Oct 30 01:05:40.901714 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 517/1033] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:05:40 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1690 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:05:41.163902 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [req-2d91644f-7e0a-4fcb-a892-8005f594d904 req-aab7beb0-b665-4fd9-ae9a-bac85666a2e6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "de41579d-0173-4dba-962d-a28c85c505de", "name": "network-vif-deleted", "tag": "c4782435-5ac9-430e-95c9-c061a2747b76"}]} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:05:41.175757 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-2d91644f-7e0a-4fcb-a892-8005f594d904 req-aab7beb0-b665-4fd9-ae9a-bac85666a2e6 service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:41.176888 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-2d91644f-7e0a-4fcb-a892-8005f594d904 req-aab7beb0-b665-4fd9-ae9a-bac85666a2e6 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:41.177401 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-2d91644f-7e0a-4fcb-a892-8005f594d904 req-aab7beb0-b665-4fd9-ae9a-bac85666a2e6 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:41.191326 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.compute.server_external_events [req-2d91644f-7e0a-4fcb-a892-8005f594d904 req-aab7beb0-b665-4fd9-ae9a-bac85666a2e6 service nova] Creating event network-vif-deleted:c4782435-5ac9-430e-95c9-c061a2747b76 for instance de41579d-0173-4dba-962d-a28c85c505de on hv-cinder-maste Oct 30 01:05:41.197727 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [req-2d91644f-7e0a-4fcb-a892-8005f594d904 req-aab7beb0-b665-4fd9-ae9a-bac85666a2e6 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037045 Oct 30 01:05:41.198472 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 517/1034] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:05:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:05:41.253703 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-c7ab9990-7bb3-4725-b9d9-6f36e9ccfc84 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "b33044ec-beab-480f-a1dc-f217b7c295f5"}} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:05:41.258840 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-c7ab9990-7bb3-4725-b9d9-6f36e9ccfc84 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: b77a1841-4f87-4b8a-96a9-a619ea67d452] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:05:41.268679 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c7ab9990-7bb3-4725-b9d9-6f36e9ccfc84 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:41.269280 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c7ab9990-7bb3-4725-b9d9-6f36e9ccfc84 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:41.270163 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c7ab9990-7bb3-4725-b9d9-6f36e9ccfc84 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:41.271744 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c7ab9990-7bb3-4725-b9d9-6f36e9ccfc84 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:41.273821 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c7ab9990-7bb3-4725-b9d9-6f36e9ccfc84 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:41.274621 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c7ab9990-7bb3-4725-b9d9-6f36e9ccfc84 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:41.608977 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-c7ab9990-7bb3-4725-b9d9-6f36e9ccfc84 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "POST /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.359372 Oct 30 01:05:41.610342 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 518/1035] 10.222.0.16 () {60 vars in 1323 bytes} [Sun Oct 30 01:05:41 2022] POST /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452/os-volume_attachments => generated 194 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:05:41.923146 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-2105f8d5-dbb4-45fd-a56d-a7761fe97b74 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:05:41.923868 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-2105f8d5-dbb4-45fd-a56d-a7761fe97b74 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:05:41.931018 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2105f8d5-dbb4-45fd-a56d-a7761fe97b74 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:41.931699 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2105f8d5-dbb4-45fd-a56d-a7761fe97b74 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:41.932483 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2105f8d5-dbb4-45fd-a56d-a7761fe97b74 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:41.934741 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2105f8d5-dbb4-45fd-a56d-a7761fe97b74 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:41.935629 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2105f8d5-dbb4-45fd-a56d-a7761fe97b74 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:41.940341 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2105f8d5-dbb4-45fd-a56d-a7761fe97b74 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:41.989863 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2105f8d5-dbb4-45fd-a56d-a7761fe97b74 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:41.990432 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2105f8d5-dbb4-45fd-a56d-a7761fe97b74 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:41.990937 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2105f8d5-dbb4-45fd-a56d-a7761fe97b74 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:41.991453 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.objects.instance [None req-2105f8d5-dbb4-45fd-a56d-a7761fe97b74 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lazy-loading 'fault' on Instance uuid de41579d-0173-4dba-962d-a28c85c505de {{(pid=118962) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 30 01:05:42.036057 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-2105f8d5-dbb4-45fd-a56d-a7761fe97b74 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:05:42.043906 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-2105f8d5-dbb4-45fd-a56d-a7761fe97b74 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 200 len: 1335 microversion: 2.1 time: 0.124969 Oct 30 01:05:42.044672 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 518/1036] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:05:41 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 1335 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:05:43.074065 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-d5aee7a7-ced2-453e-9bd1-dd4cae860f88 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:05:43.075204 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-d5aee7a7-ced2-453e-9bd1-dd4cae860f88 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: de41579d-0173-4dba-962d-a28c85c505de] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:05:43.086001 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d5aee7a7-ced2-453e-9bd1-dd4cae860f88 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:43.086797 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d5aee7a7-ced2-453e-9bd1-dd4cae860f88 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:43.087522 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d5aee7a7-ced2-453e-9bd1-dd4cae860f88 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:43.088946 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d5aee7a7-ced2-453e-9bd1-dd4cae860f88 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:43.089405 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d5aee7a7-ced2-453e-9bd1-dd4cae860f88 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:43.089816 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d5aee7a7-ced2-453e-9bd1-dd4cae860f88 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:43.103123 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.wsgi [None req-d5aee7a7-ced2-453e-9bd1-dd4cae860f88 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] HTTP exception thrown: Instance de41579d-0173-4dba-962d-a28c85c505de could not be found. Oct 30 01:05:43.104779 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-d5aee7a7-ced2-453e-9bd1-dd4cae860f88 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Request method failure captured: Oct 30 01:05:43.104779 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: request: GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de HTTP/1.1 Oct 30 01:05:43.104779 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Accept: application/json Oct 30 01:05:43.104779 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Accept-Encoding: identity Oct 30 01:05:43.104779 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Connection: close Oct 30 01:05:43.104779 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Content-Length: 0 Oct 30 01:05:43.104779 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Content-Type: application/json Oct 30 01:05:43.104779 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Host: 10.222.0.16 Oct 30 01:05:43.104779 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Openstack-System-Scope: None Oct 30 01:05:43.104779 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: User-Agent: python-urllib3/1.26.12 Oct 30 01:05:43.104779 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Auth-Token: gAAAAABjXcwtPy0eLS3MC3-cmfmCjFSfeavovRIYXfEfNVAywVbIQ30ctEe5V4sGvi1A3M4sDeUVrGvF9i6stAWwO3TejIedmS8RLhBtx-h_JsINfLnyH884rCMd4-euC7isoMsjNUbUBUpUbdHwx6JXx5oDJcwXMr1RZh9X14AJM7k_e6z6bu8 Oct 30 01:05:43.104779 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Domain-Id: None Oct 30 01:05:43.104779 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Domain-Name: None Oct 30 01:05:43.104779 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Identity-Status: Confirmed Oct 30 01:05:43.104779 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Is-Admin-Project: True Oct 30 01:05:43.104779 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Domain-Id: default Oct 30 01:05:43.104779 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Domain-Name: Default Oct 30 01:05:43.104779 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Id: f48276ac51dd4bb1912fc670433cf6c4 Oct 30 01:05:43.104779 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-764720217 Oct 30 01:05:43.104779 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Role: reader,member Oct 30 01:05:43.104779 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Roles: reader,member Oct 30 01:05:43.106377 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_f48276ac51dd4bb1912fc670433cf6c4"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f48276ac51dd4bb1912fc670433cf6c4"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/f48276ac51dd4bb1912fc670433cf6c4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f48276ac51dd4bb1912fc670433cf6c4"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 01:05:43.106377 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant: tempest-ServerRescueNegativeTestJSON-764720217 Oct 30 01:05:43.106377 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant-Id: f48276ac51dd4bb1912fc670433cf6c4 Oct 30 01:05:43.106377 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-764720217 Oct 30 01:05:43.106377 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User: tempest-ServerRescueNegativeTestJSON-764720217-project Oct 30 01:05:43.106377 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Domain-Id: default Oct 30 01:05:43.106377 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Domain-Name: Default Oct 30 01:05:43.106377 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Id: 31b7342ecf07408cab6bb040f361fa1f Oct 30 01:05:43.106377 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Name: tempest-ServerRescueNegativeTestJSON-764720217-project Oct 30 01:05:43.106377 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: method: > Oct 30 01:05:43.106377 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: exception: Instance de41579d-0173-4dba-962d-a28c85c505de could not be found. Oct 30 01:05:43.106377 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: action_args: {'id': 'de41579d-0173-4dba-962d-a28c85c505de'} Oct 30 01:05:43.107329 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 01:05:43.107329 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:05:43.107329 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 30 01:05:43.107329 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 30 01:05:43.107329 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 30 01:05:43.107329 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 30 01:05:43.107329 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 30 01:05:43.107329 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 30 01:05:43.107329 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 30 01:05:43.107329 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 30 01:05:43.107329 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance de41579d-0173-4dba-962d-a28c85c505de could not be found. Oct 30 01:05:43.107329 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:05:43.107329 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:05:43.107329 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:05:43.107329 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:05:43.107329 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:05:43.107329 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:05:43.107329 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 01:05:43.107329 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 01:05:43.107329 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 01:05:43.109004 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 01:05:43.109004 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 30 01:05:43.109004 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 30 01:05:43.109004 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 30 01:05:43.109004 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 30 01:05:43.109004 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 30 01:05:43.109004 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 30 01:05:43.109004 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance de41579d-0173-4dba-962d-a28c85c505de could not be found. Oct 30 01:05:43.109004 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:05:43.109004 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:05:43.109004 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:05:43.109004 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:05:43.109004 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:05:43.109004 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:05:43.109004 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 01:05:43.109004 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 01:05:43.109004 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance de41579d-0173-4dba-962d-a28c85c505de could not be found. Oct 30 01:05:43.109004 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:05:43.110625 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-d5aee7a7-ced2-453e-9bd1-dd4cae860f88 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Returning 404 to user: Instance de41579d-0173-4dba-962d-a28c85c505de could not be found. {{(pid=118961) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 01:05:43.110625 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-d5aee7a7-ced2-453e-9bd1-dd4cae860f88 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de" status: 404 len: 111 microversion: 2.1 time: 0.040127 Oct 30 01:05:43.110625 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 519/1037] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:05:43 2022] GET /compute/v2.1/servers/de41579d-0173-4dba-962d-a28c85c505de => generated 111 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 30 01:05:43.117666 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-d61f0de6-4414-4b1e-826c-1d72ac55f2af tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '.version_select at 0x7f0eafc3e940>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:05:43.142779 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-d61f0de6-4414-4b1e-826c-1d72ac55f2af tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1855286619" status: 202 len: 0 microversion: 2.1 time: 0.031384 Oct 30 01:05:43.143395 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 519/1038] 10.222.0.16 () {58 vars in 1221 bytes} [Sun Oct 30 01:05:43 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1855286619 => generated 0 bytes in 33 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 30 01:05:46.068249 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-c3b31ac7-9016-4719-9a27-cd07d25108c2 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:05:46.068829 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-c3b31ac7-9016-4719-9a27-cd07d25108c2 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:05:46.076061 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c3b31ac7-9016-4719-9a27-cd07d25108c2 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:46.076547 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c3b31ac7-9016-4719-9a27-cd07d25108c2 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:46.077071 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c3b31ac7-9016-4719-9a27-cd07d25108c2 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:46.077772 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c3b31ac7-9016-4719-9a27-cd07d25108c2 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:46.078221 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c3b31ac7-9016-4719-9a27-cd07d25108c2 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:46.078639 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c3b31ac7-9016-4719-9a27-cd07d25108c2 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:46.108765 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-c3b31ac7-9016-4719-9a27-cd07d25108c2 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Going to try to terminate instance {{(pid=118961) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 30 01:05:46.204847 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-c3b31ac7-9016-4719-9a27-cd07d25108c2 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "DELETE /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 204 len: 0 microversion: 2.1 time: 0.142178 Oct 30 01:05:46.205392 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 520/1039] 10.222.0.16 () {58 vars in 1239 bytes} [Sun Oct 30 01:05:46 2022] DELETE /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 30 01:05:46.215254 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-78951ea4-27cd-45fe-859c-88316e0efa97 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:05:46.215889 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-78951ea4-27cd-45fe-859c-88316e0efa97 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:05:46.223752 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-78951ea4-27cd-45fe-859c-88316e0efa97 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:46.224299 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-78951ea4-27cd-45fe-859c-88316e0efa97 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:46.224757 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-78951ea4-27cd-45fe-859c-88316e0efa97 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:46.225449 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-78951ea4-27cd-45fe-859c-88316e0efa97 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:46.225889 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-78951ea4-27cd-45fe-859c-88316e0efa97 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:46.226302 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-78951ea4-27cd-45fe-859c-88316e0efa97 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:46.373494 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-78951ea4-27cd-45fe-859c-88316e0efa97 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:05:46.382713 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-78951ea4-27cd-45fe-859c-88316e0efa97 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1508 microversion: 2.1 time: 0.172716 Oct 30 01:05:46.383291 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 520/1040] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:05:46 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1508 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:05:47.406303 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-ad9c72cd-a303-41c1-80a1-b864df023e4c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:05:47.407151 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-ad9c72cd-a303-41c1-80a1-b864df023e4c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:05:47.414116 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ad9c72cd-a303-41c1-80a1-b864df023e4c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:47.414610 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ad9c72cd-a303-41c1-80a1-b864df023e4c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:47.415107 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ad9c72cd-a303-41c1-80a1-b864df023e4c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:47.415849 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ad9c72cd-a303-41c1-80a1-b864df023e4c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:47.416302 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ad9c72cd-a303-41c1-80a1-b864df023e4c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:47.416726 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ad9c72cd-a303-41c1-80a1-b864df023e4c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:47.660077 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-ad9c72cd-a303-41c1-80a1-b864df023e4c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:05:47.668520 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-ad9c72cd-a303-41c1-80a1-b864df023e4c tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1508 microversion: 2.1 time: 0.266540 Oct 30 01:05:47.669095 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 521/1041] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:05:47 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1508 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:05:47.901412 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [req-104d5010-07e1-48fa-b0ca-22f449ee99fd req-29e76647-b5ee-44c5-bac2-11fafb515665 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "60b2f6e0-a160-4d3f-b036-38597b60da68", "name": "network-vif-deleted", "tag": "88fd74d2-892d-4aaf-b602-415c64c5b9aa"}]} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:05:47.911356 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-104d5010-07e1-48fa-b0ca-22f449ee99fd req-29e76647-b5ee-44c5-bac2-11fafb515665 service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:47.912229 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-104d5010-07e1-48fa-b0ca-22f449ee99fd req-29e76647-b5ee-44c5-bac2-11fafb515665 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:47.912229 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-104d5010-07e1-48fa-b0ca-22f449ee99fd req-29e76647-b5ee-44c5-bac2-11fafb515665 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:47.923391 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.compute.server_external_events [req-104d5010-07e1-48fa-b0ca-22f449ee99fd req-29e76647-b5ee-44c5-bac2-11fafb515665 service nova] Creating event network-vif-deleted:88fd74d2-892d-4aaf-b602-415c64c5b9aa for instance 60b2f6e0-a160-4d3f-b036-38597b60da68 on hv-cinder-maste Oct 30 01:05:47.928456 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [req-104d5010-07e1-48fa-b0ca-22f449ee99fd req-29e76647-b5ee-44c5-bac2-11fafb515665 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030739 Oct 30 01:05:47.929063 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 521/1042] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:05:47 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:05:48.692894 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-be97180c-a571-4da4-84a0-8891ecacfa73 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:05:48.693447 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-be97180c-a571-4da4-84a0-8891ecacfa73 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:05:48.700625 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-be97180c-a571-4da4-84a0-8891ecacfa73 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:48.701226 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-be97180c-a571-4da4-84a0-8891ecacfa73 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:48.701709 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-be97180c-a571-4da4-84a0-8891ecacfa73 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:48.702784 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-be97180c-a571-4da4-84a0-8891ecacfa73 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:48.703366 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-be97180c-a571-4da4-84a0-8891ecacfa73 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:48.703840 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-be97180c-a571-4da4-84a0-8891ecacfa73 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:48.739511 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-be97180c-a571-4da4-84a0-8891ecacfa73 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:48.740025 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-be97180c-a571-4da4-84a0-8891ecacfa73 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:48.740514 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-be97180c-a571-4da4-84a0-8891ecacfa73 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:48.740966 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.objects.instance [None req-be97180c-a571-4da4-84a0-8891ecacfa73 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lazy-loading 'fault' on Instance uuid 60b2f6e0-a160-4d3f-b036-38597b60da68 {{(pid=118961) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 30 01:05:48.787159 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-be97180c-a571-4da4-84a0-8891ecacfa73 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:05:48.799019 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-be97180c-a571-4da4-84a0-8891ecacfa73 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 200 len: 1311 microversion: 2.1 time: 0.110101 Oct 30 01:05:48.799872 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 522/1043] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:05:48 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 1311 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:05:49.233313 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-10bd9d9e-80a2-4d93-9f1c-b28275034c67 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "b33044ec-beab-480f-a1dc-f217b7c295f5"}} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:05:49.235829 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-10bd9d9e-80a2-4d93-9f1c-b28275034c67 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: b77a1841-4f87-4b8a-96a9-a619ea67d452] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:05:49.241905 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-10bd9d9e-80a2-4d93-9f1c-b28275034c67 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:49.242376 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-10bd9d9e-80a2-4d93-9f1c-b28275034c67 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:49.242795 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-10bd9d9e-80a2-4d93-9f1c-b28275034c67 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:49.243470 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-10bd9d9e-80a2-4d93-9f1c-b28275034c67 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:49.243948 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-10bd9d9e-80a2-4d93-9f1c-b28275034c67 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:49.244474 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-10bd9d9e-80a2-4d93-9f1c-b28275034c67 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:49.325233 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.wsgi [None req-10bd9d9e-80a2-4d93-9f1c-b28275034c67 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] HTTP exception thrown: Invalid volume: volume b33044ec-beab-480f-a1dc-f217b7c295f5 is already attached to instances: b77a1841-4f87-4b8a-96a9-a619ea67d452 Oct 30 01:05:49.327212 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-10bd9d9e-80a2-4d93-9f1c-b28275034c67 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Request method failure captured: Oct 30 01:05:49.327212 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: request: POST /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452/os-volume_attachments HTTP/1.1 Oct 30 01:05:49.327212 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept: application/json Oct 30 01:05:49.327212 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept-Encoding: identity Oct 30 01:05:49.327212 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Connection: close Oct 30 01:05:49.327212 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Length: 74 Oct 30 01:05:49.327212 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Type: application/json Oct 30 01:05:49.327212 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Host: 10.222.0.16 Oct 30 01:05:49.327212 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Openstack-System-Scope: None Oct 30 01:05:49.327212 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: User-Agent: python-urllib3/1.26.12 Oct 30 01:05:49.327212 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Auth-Token: gAAAAABjXcuedsaRjtnPRcHo4HqTnifOYe1vcHHjSzl7t8SsEKfOt41JBob3t9Ien606ucEZmcgmpf5H0lzlL6oZS4vLpZERLur9y2YRhqd9AYPyhbrmUkrYmKAssv6PSCysf6z3-myn1YKr62Nxqfs7t1lZGZnZuH_I2j8LHl5PnVXWJxyADCY Oct 30 01:05:49.327212 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Id: None Oct 30 01:05:49.327212 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Name: None Oct 30 01:05:49.327212 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Identity-Status: Confirmed Oct 30 01:05:49.327212 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Is-Admin-Project: True Oct 30 01:05:49.327212 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Id: default Oct 30 01:05:49.327212 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Name: Default Oct 30 01:05:49.327212 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Id: 6b68b886c19c45d585409651f33b8626 Oct 30 01:05:49.327212 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Name: tempest-AttachVolumeNegativeTest-618009204 Oct 30 01:05:49.327212 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Role: reader,member Oct 30 01:05:49.327212 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Roles: reader,member Oct 30 01:05:49.328921 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_6b68b886c19c45d585409651f33b8626"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6b68b886c19c45d585409651f33b8626"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/6b68b886c19c45d585409651f33b8626"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6b68b886c19c45d585409651f33b8626"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 01:05:49.328921 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant: tempest-AttachVolumeNegativeTest-618009204 Oct 30 01:05:49.328921 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Id: 6b68b886c19c45d585409651f33b8626 Oct 30 01:05:49.328921 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-618009204 Oct 30 01:05:49.328921 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User: tempest-AttachVolumeNegativeTest-618009204-project Oct 30 01:05:49.328921 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Id: default Oct 30 01:05:49.328921 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Name: Default Oct 30 01:05:49.328921 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Id: ecaf8ec105c04b919256f852da859eba Oct 30 01:05:49.328921 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Name: tempest-AttachVolumeNegativeTest-618009204-project Oct 30 01:05:49.328921 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Oct 30 01:05:49.328921 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: {"volumeAttachment": {"volumeId": "b33044ec-beab-480f-a1dc-f217b7c295f5"}} Oct 30 01:05:49.328921 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: method: > Oct 30 01:05:49.329798 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: exception: Invalid volume: volume b33044ec-beab-480f-a1dc-f217b7c295f5 is already attached to instances: b77a1841-4f87-4b8a-96a9-a619ea67d452 Oct 30 01:05:49.329798 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: action_args: {'server_id': 'b77a1841-4f87-4b8a-96a9-a619ea67d452', 'body': {'volumeAttachment': {'volumeId': 'b33044ec-beab-480f-a1dc-f217b7c295f5'}}} Oct 30 01:05:49.329798 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 01:05:49.329798 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:05:49.329798 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Oct 30 01:05:49.329798 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Oct 30 01:05:49.329798 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 30 01:05:49.329798 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 30 01:05:49.329798 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Oct 30 01:05:49.329798 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Oct 30 01:05:49.329798 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5130, in attach_volume Oct 30 01:05:49.329798 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Oct 30 01:05:49.329798 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4970, in _check_volume_already_attached Oct 30 01:05:49.329798 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Oct 30 01:05:49.329798 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume b33044ec-beab-480f-a1dc-f217b7c295f5 is already attached to instances: b77a1841-4f87-4b8a-96a9-a619ea67d452 Oct 30 01:05:49.329798 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:05:49.329798 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:05:49.329798 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:05:49.329798 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:05:49.331196 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:05:49.331196 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:05:49.331196 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 01:05:49.331196 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 01:05:49.331196 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 01:05:49.331196 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 01:05:49.331196 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 30 01:05:49.331196 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 30 01:05:49.331196 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 30 01:05:49.331196 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 30 01:05:49.331196 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 30 01:05:49.331196 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 30 01:05:49.331196 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Oct 30 01:05:49.331196 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Oct 30 01:05:49.331196 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume b33044ec-beab-480f-a1dc-f217b7c295f5 is already attached to instances: b77a1841-4f87-4b8a-96a9-a619ea67d452 Oct 30 01:05:49.331196 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:05:49.331196 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:05:49.331196 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:05:49.331196 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:05:49.331196 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:05:49.332861 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:05:49.332861 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 01:05:49.332861 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 01:05:49.332861 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume b33044ec-beab-480f-a1dc-f217b7c295f5 is already attached to instances: b77a1841-4f87-4b8a-96a9-a619ea67d452 Oct 30 01:05:49.332861 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:05:49.332861 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-10bd9d9e-80a2-4d93-9f1c-b28275034c67 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Returning 400 to user: Invalid volume: volume b33044ec-beab-480f-a1dc-f217b7c295f5 is already attached to instances: b77a1841-4f87-4b8a-96a9-a619ea67d452 {{(pid=118962) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 01:05:49.332861 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-10bd9d9e-80a2-4d93-9f1c-b28275034c67 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "POST /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.100409 Oct 30 01:05:49.332861 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 522/1044] 10.222.0.16 () {60 vars in 1323 bytes} [Sun Oct 30 01:05:49 2022] POST /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452/os-volume_attachments => generated 174 bytes in 102 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 30 01:05:49.398543 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-f63beb2d-f111-4f4b-bed6-ccdcbced1420 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:05:49.399048 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-f63beb2d-f111-4f4b-bed6-ccdcbced1420 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: b77a1841-4f87-4b8a-96a9-a619ea67d452] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:05:49.405173 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f63beb2d-f111-4f4b-bed6-ccdcbced1420 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:49.405660 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f63beb2d-f111-4f4b-bed6-ccdcbced1420 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:49.406143 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f63beb2d-f111-4f4b-bed6-ccdcbced1420 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:49.406939 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f63beb2d-f111-4f4b-bed6-ccdcbced1420 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:49.407396 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f63beb2d-f111-4f4b-bed6-ccdcbced1420 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:49.407848 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f63beb2d-f111-4f4b-bed6-ccdcbced1420 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:49.426020 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell1:SELECT=92,nova_cell1:INSERT=4,nova_cell1:UPDATE=4 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:05:49.497846 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.objects.instance [None req-f63beb2d-f111-4f4b-bed6-ccdcbced1420 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lazy-loading 'services' on Instance uuid b77a1841-4f87-4b8a-96a9-a619ea67d452 {{(pid=118961) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 30 01:05:49.578312 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-f63beb2d-f111-4f4b-bed6-ccdcbced1420 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "DELETE /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452/os-volume_attachments/b33044ec-beab-480f-a1dc-f217b7c295f5" status: 202 len: 0 microversion: 2.1 time: 0.183476 Oct 30 01:05:49.579453 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 523/1045] 10.222.0.16 () {58 vars in 1416 bytes} [Sun Oct 30 01:05:49 2022] DELETE /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452/os-volume_attachments/b33044ec-beab-480f-a1dc-f217b7c295f5 => generated 0 bytes in 185 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 30 01:05:49.588476 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-2929d81c-5c50-49cd-bf66-fa9d4f67963b tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:05:49.590817 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-2929d81c-5c50-49cd-bf66-fa9d4f67963b tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: b77a1841-4f87-4b8a-96a9-a619ea67d452] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:05:49.606398 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2929d81c-5c50-49cd-bf66-fa9d4f67963b tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:49.607174 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2929d81c-5c50-49cd-bf66-fa9d4f67963b tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:49.607613 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2929d81c-5c50-49cd-bf66-fa9d4f67963b tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:49.608429 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2929d81c-5c50-49cd-bf66-fa9d4f67963b tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:49.608885 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2929d81c-5c50-49cd-bf66-fa9d4f67963b tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:49.609473 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2929d81c-5c50-49cd-bf66-fa9d4f67963b tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:49.641815 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-2929d81c-5c50-49cd-bf66-fa9d4f67963b tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057861 Oct 30 01:05:49.647146 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 523/1046] 10.222.0.16 () {58 vars in 1302 bytes} [Sun Oct 30 01:05:49 2022] GET /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:05:49.828729 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-fde9165e-3aad-4825-8838-0b06b7975bfe tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:05:49.829435 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-fde9165e-3aad-4825-8838-0b06b7975bfe tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: 60b2f6e0-a160-4d3f-b036-38597b60da68] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:05:49.836088 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-fde9165e-3aad-4825-8838-0b06b7975bfe tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:49.836571 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-fde9165e-3aad-4825-8838-0b06b7975bfe tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:49.837011 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-fde9165e-3aad-4825-8838-0b06b7975bfe tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:49.837699 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-fde9165e-3aad-4825-8838-0b06b7975bfe tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:49.838479 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-fde9165e-3aad-4825-8838-0b06b7975bfe tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:49.838901 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-fde9165e-3aad-4825-8838-0b06b7975bfe tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:49.852498 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.wsgi [None req-fde9165e-3aad-4825-8838-0b06b7975bfe tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] HTTP exception thrown: Instance 60b2f6e0-a160-4d3f-b036-38597b60da68 could not be found. Oct 30 01:05:49.854120 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-fde9165e-3aad-4825-8838-0b06b7975bfe tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Request method failure captured: Oct 30 01:05:49.854120 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: request: GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 HTTP/1.1 Oct 30 01:05:49.854120 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Accept: application/json Oct 30 01:05:49.854120 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Accept-Encoding: identity Oct 30 01:05:49.854120 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Connection: close Oct 30 01:05:49.854120 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Content-Length: 0 Oct 30 01:05:49.854120 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Content-Type: application/json Oct 30 01:05:49.854120 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Host: 10.222.0.16 Oct 30 01:05:49.854120 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Openstack-System-Scope: None Oct 30 01:05:49.854120 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: User-Agent: python-urllib3/1.26.12 Oct 30 01:05:49.854120 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Auth-Token: gAAAAABjXcwtPy0eLS3MC3-cmfmCjFSfeavovRIYXfEfNVAywVbIQ30ctEe5V4sGvi1A3M4sDeUVrGvF9i6stAWwO3TejIedmS8RLhBtx-h_JsINfLnyH884rCMd4-euC7isoMsjNUbUBUpUbdHwx6JXx5oDJcwXMr1RZh9X14AJM7k_e6z6bu8 Oct 30 01:05:49.854120 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Domain-Id: None Oct 30 01:05:49.854120 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Domain-Name: None Oct 30 01:05:49.854120 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Identity-Status: Confirmed Oct 30 01:05:49.854120 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Is-Admin-Project: True Oct 30 01:05:49.854120 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Domain-Id: default Oct 30 01:05:49.854120 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Domain-Name: Default Oct 30 01:05:49.854120 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Id: f48276ac51dd4bb1912fc670433cf6c4 Oct 30 01:05:49.854120 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-764720217 Oct 30 01:05:49.854120 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Role: reader,member Oct 30 01:05:49.854120 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Roles: reader,member Oct 30 01:05:49.855679 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_f48276ac51dd4bb1912fc670433cf6c4"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f48276ac51dd4bb1912fc670433cf6c4"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/f48276ac51dd4bb1912fc670433cf6c4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f48276ac51dd4bb1912fc670433cf6c4"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 01:05:49.855679 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant: tempest-ServerRescueNegativeTestJSON-764720217 Oct 30 01:05:49.855679 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant-Id: f48276ac51dd4bb1912fc670433cf6c4 Oct 30 01:05:49.855679 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-764720217 Oct 30 01:05:49.855679 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User: tempest-ServerRescueNegativeTestJSON-764720217-project Oct 30 01:05:49.855679 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Domain-Id: default Oct 30 01:05:49.855679 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Domain-Name: Default Oct 30 01:05:49.855679 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Id: 31b7342ecf07408cab6bb040f361fa1f Oct 30 01:05:49.855679 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Name: tempest-ServerRescueNegativeTestJSON-764720217-project Oct 30 01:05:49.855679 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: method: > Oct 30 01:05:49.855679 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: exception: Instance 60b2f6e0-a160-4d3f-b036-38597b60da68 could not be found. Oct 30 01:05:49.855679 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: action_args: {'id': '60b2f6e0-a160-4d3f-b036-38597b60da68'} Oct 30 01:05:49.856630 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 01:05:49.856630 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:05:49.856630 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 30 01:05:49.856630 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 30 01:05:49.856630 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 30 01:05:49.856630 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 30 01:05:49.856630 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 30 01:05:49.856630 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 30 01:05:49.856630 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 30 01:05:49.856630 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 30 01:05:49.856630 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 60b2f6e0-a160-4d3f-b036-38597b60da68 could not be found. Oct 30 01:05:49.856630 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:05:49.856630 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:05:49.856630 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:05:49.856630 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:05:49.856630 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:05:49.856630 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:05:49.856630 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 01:05:49.856630 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 01:05:49.856630 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 01:05:49.858169 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 01:05:49.858169 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 30 01:05:49.858169 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 30 01:05:49.858169 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 30 01:05:49.858169 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 30 01:05:49.858169 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 30 01:05:49.858169 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 30 01:05:49.858169 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 60b2f6e0-a160-4d3f-b036-38597b60da68 could not be found. Oct 30 01:05:49.858169 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:05:49.858169 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:05:49.858169 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:05:49.858169 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:05:49.858169 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:05:49.858169 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:05:49.858169 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 01:05:49.858169 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 01:05:49.858169 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 60b2f6e0-a160-4d3f-b036-38597b60da68 could not be found. Oct 30 01:05:49.858169 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:05:49.859628 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-fde9165e-3aad-4825-8838-0b06b7975bfe tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Returning 404 to user: Instance 60b2f6e0-a160-4d3f-b036-38597b60da68 could not be found. {{(pid=118961) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 01:05:49.859628 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-fde9165e-3aad-4825-8838-0b06b7975bfe tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68" status: 404 len: 111 microversion: 2.1 time: 0.032234 Oct 30 01:05:49.859628 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 524/1047] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:05:49 2022] GET /compute/v2.1/servers/60b2f6e0-a160-4d3f-b036-38597b60da68 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 30 01:05:49.864412 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-3645f35e-b947-4467-9e02-306f478dab22 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:05:49.864901 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-3645f35e-b947-4467-9e02-306f478dab22 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: c7be44ac-3b42-4ace-ac56-65c117e839c6] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:05:49.870893 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3645f35e-b947-4467-9e02-306f478dab22 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:49.871364 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3645f35e-b947-4467-9e02-306f478dab22 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:49.871808 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3645f35e-b947-4467-9e02-306f478dab22 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:49.872491 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3645f35e-b947-4467-9e02-306f478dab22 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:49.872934 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3645f35e-b947-4467-9e02-306f478dab22 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:49.873344 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3645f35e-b947-4467-9e02-306f478dab22 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:49.898775 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-3645f35e-b947-4467-9e02-306f478dab22 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: c7be44ac-3b42-4ace-ac56-65c117e839c6] Going to try to terminate instance {{(pid=118962) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 30 01:05:49.980004 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-3645f35e-b947-4467-9e02-306f478dab22 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "DELETE /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6" status: 204 len: 0 microversion: 2.1 time: 0.119270 Oct 30 01:05:49.980580 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 524/1048] 10.222.0.16 () {58 vars in 1239 bytes} [Sun Oct 30 01:05:49 2022] DELETE /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 30 01:05:49.989222 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-bde1a566-d567-4d33-9f05-4559fe2b54f1 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:05:49.989858 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-bde1a566-d567-4d33-9f05-4559fe2b54f1 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: c7be44ac-3b42-4ace-ac56-65c117e839c6] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:05:49.996219 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bde1a566-d567-4d33-9f05-4559fe2b54f1 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:49.996699 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bde1a566-d567-4d33-9f05-4559fe2b54f1 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:49.997124 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bde1a566-d567-4d33-9f05-4559fe2b54f1 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:49.997811 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bde1a566-d567-4d33-9f05-4559fe2b54f1 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:49.998252 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bde1a566-d567-4d33-9f05-4559fe2b54f1 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:49.998679 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bde1a566-d567-4d33-9f05-4559fe2b54f1 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:50.138307 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-bde1a566-d567-4d33-9f05-4559fe2b54f1 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:05:50.146568 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-bde1a566-d567-4d33-9f05-4559fe2b54f1 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6" status: 200 len: 1522 microversion: 2.1 time: 0.161258 Oct 30 01:05:50.147312 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 525/1049] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:05:49 2022] GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6 => generated 1522 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 01:05:50.659304 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-07ee5b26-f49b-4863-a8aa-ece2b1a1d493 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:05:50.662568 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-07ee5b26-f49b-4863-a8aa-ece2b1a1d493 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: b77a1841-4f87-4b8a-96a9-a619ea67d452] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:05:50.672751 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-07ee5b26-f49b-4863-a8aa-ece2b1a1d493 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:50.673225 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-07ee5b26-f49b-4863-a8aa-ece2b1a1d493 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:50.673645 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-07ee5b26-f49b-4863-a8aa-ece2b1a1d493 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:50.674334 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-07ee5b26-f49b-4863-a8aa-ece2b1a1d493 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:50.674767 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-07ee5b26-f49b-4863-a8aa-ece2b1a1d493 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:50.675178 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-07ee5b26-f49b-4863-a8aa-ece2b1a1d493 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:50.705635 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-07ee5b26-f49b-4863-a8aa-ece2b1a1d493 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053321 Oct 30 01:05:50.706190 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 525/1050] 10.222.0.16 () {58 vars in 1302 bytes} [Sun Oct 30 01:05:50 2022] GET /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:05:51.172653 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-deafd1b9-40a0-4684-a04f-5819b18ae759 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:05:51.173493 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-deafd1b9-40a0-4684-a04f-5819b18ae759 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: c7be44ac-3b42-4ace-ac56-65c117e839c6] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:05:51.183043 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-deafd1b9-40a0-4684-a04f-5819b18ae759 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:51.183894 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-deafd1b9-40a0-4684-a04f-5819b18ae759 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:51.184714 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-deafd1b9-40a0-4684-a04f-5819b18ae759 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:51.185797 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-deafd1b9-40a0-4684-a04f-5819b18ae759 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:51.186531 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-deafd1b9-40a0-4684-a04f-5819b18ae759 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:51.187393 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-deafd1b9-40a0-4684-a04f-5819b18ae759 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:51.331699 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-deafd1b9-40a0-4684-a04f-5819b18ae759 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:05:51.340244 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-deafd1b9-40a0-4684-a04f-5819b18ae759 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6" status: 200 len: 1522 microversion: 2.1 time: 0.173641 Oct 30 01:05:51.340794 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 526/1051] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:05:51 2022] GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6 => generated 1522 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:05:51.720237 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-057ee171-a85c-4bc1-9929-def1d9cb782a tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:05:51.723139 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-057ee171-a85c-4bc1-9929-def1d9cb782a tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: b77a1841-4f87-4b8a-96a9-a619ea67d452] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:05:51.730780 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-057ee171-a85c-4bc1-9929-def1d9cb782a tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:51.731299 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-057ee171-a85c-4bc1-9929-def1d9cb782a tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:51.731710 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-057ee171-a85c-4bc1-9929-def1d9cb782a tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:51.732445 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-057ee171-a85c-4bc1-9929-def1d9cb782a tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:51.732892 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-057ee171-a85c-4bc1-9929-def1d9cb782a tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:51.733313 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-057ee171-a85c-4bc1-9929-def1d9cb782a tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:51.764667 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-057ee171-a85c-4bc1-9929-def1d9cb782a tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.048761 Oct 30 01:05:51.765339 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 526/1052] 10.222.0.16 () {58 vars in 1302 bytes} [Sun Oct 30 01:05:51 2022] GET /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452/os-volume_attachments => generated 25 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 30 01:05:51.835628 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [req-67d82770-cdd1-45d0-98e5-f49e5a0859ad req-1e72ebfc-4eb3-4293-beb2-c7240cd2e6f8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c7be44ac-3b42-4ace-ac56-65c117e839c6", "name": "network-vif-deleted", "tag": "2a5ef263-4503-4e15-8a9c-3e1f7cdcf464"}]} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:05:51.842239 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-a1d0d906-376f-4731-a3f0-2ad27fbd25fa tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:05:51.842805 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-a1d0d906-376f-4731-a3f0-2ad27fbd25fa tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: b77a1841-4f87-4b8a-96a9-a619ea67d452] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:05:51.845970 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-67d82770-cdd1-45d0-98e5-f49e5a0859ad req-1e72ebfc-4eb3-4293-beb2-c7240cd2e6f8 service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:51.846452 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-67d82770-cdd1-45d0-98e5-f49e5a0859ad req-1e72ebfc-4eb3-4293-beb2-c7240cd2e6f8 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:51.846881 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-67d82770-cdd1-45d0-98e5-f49e5a0859ad req-1e72ebfc-4eb3-4293-beb2-c7240cd2e6f8 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:51.850440 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a1d0d906-376f-4731-a3f0-2ad27fbd25fa tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:51.850929 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a1d0d906-376f-4731-a3f0-2ad27fbd25fa tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:51.852097 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a1d0d906-376f-4731-a3f0-2ad27fbd25fa tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:51.852865 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a1d0d906-376f-4731-a3f0-2ad27fbd25fa tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:51.853620 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a1d0d906-376f-4731-a3f0-2ad27fbd25fa tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:51.854661 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a1d0d906-376f-4731-a3f0-2ad27fbd25fa tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:51.859963 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.compute.server_external_events [req-67d82770-cdd1-45d0-98e5-f49e5a0859ad req-1e72ebfc-4eb3-4293-beb2-c7240cd2e6f8 service nova] Creating event network-vif-deleted:2a5ef263-4503-4e15-8a9c-3e1f7cdcf464 for instance c7be44ac-3b42-4ace-ac56-65c117e839c6 on hv-cinder-maste Oct 30 01:05:51.865770 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [req-67d82770-cdd1-45d0-98e5-f49e5a0859ad req-1e72ebfc-4eb3-4293-beb2-c7240cd2e6f8 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040126 Oct 30 01:05:51.867164 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 527/1053] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:05:51 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:05:51.885530 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-a1d0d906-376f-4731-a3f0-2ad27fbd25fa tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: b77a1841-4f87-4b8a-96a9-a619ea67d452] Going to try to terminate instance {{(pid=118962) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 30 01:05:51.936830 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:05:51.971604 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-a1d0d906-376f-4731-a3f0-2ad27fbd25fa tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "DELETE /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452" status: 204 len: 0 microversion: 2.1 time: 0.133025 Oct 30 01:05:51.972461 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 527/1054] 10.222.0.16 () {58 vars in 1239 bytes} [Sun Oct 30 01:05:51 2022] DELETE /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Oct 30 01:05:51.982075 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-579e6ce5-f3d4-49d9-b0c3-80c27b14c308 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:05:51.983182 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-579e6ce5-f3d4-49d9-b0c3-80c27b14c308 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: b77a1841-4f87-4b8a-96a9-a619ea67d452] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:05:51.990446 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-579e6ce5-f3d4-49d9-b0c3-80c27b14c308 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:51.990973 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-579e6ce5-f3d4-49d9-b0c3-80c27b14c308 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:51.991374 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-579e6ce5-f3d4-49d9-b0c3-80c27b14c308 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:51.992331 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-579e6ce5-f3d4-49d9-b0c3-80c27b14c308 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:51.992785 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-579e6ce5-f3d4-49d9-b0c3-80c27b14c308 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:51.993384 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-579e6ce5-f3d4-49d9-b0c3-80c27b14c308 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:52.128355 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-579e6ce5-f3d4-49d9-b0c3-80c27b14c308 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:05:52.136787 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-579e6ce5-f3d4-49d9-b0c3-80c27b14c308 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452" status: 200 len: 1681 microversion: 2.1 time: 0.159209 Oct 30 01:05:52.137353 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 528/1055] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:05:51 2022] GET /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452 => generated 1681 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 01:05:52.365931 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-21592fe7-3e50-4afa-bb13-daac2db58218 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:05:52.366624 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-21592fe7-3e50-4afa-bb13-daac2db58218 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: c7be44ac-3b42-4ace-ac56-65c117e839c6] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:05:52.373850 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-21592fe7-3e50-4afa-bb13-daac2db58218 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:52.374382 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-21592fe7-3e50-4afa-bb13-daac2db58218 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:52.374858 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-21592fe7-3e50-4afa-bb13-daac2db58218 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:52.375623 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-21592fe7-3e50-4afa-bb13-daac2db58218 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:52.376425 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-21592fe7-3e50-4afa-bb13-daac2db58218 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:52.376827 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-21592fe7-3e50-4afa-bb13-daac2db58218 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:52.407974 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-21592fe7-3e50-4afa-bb13-daac2db58218 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:52.408455 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-21592fe7-3e50-4afa-bb13-daac2db58218 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:52.408884 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-21592fe7-3e50-4afa-bb13-daac2db58218 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:52.409320 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.objects.instance [None req-21592fe7-3e50-4afa-bb13-daac2db58218 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lazy-loading 'fault' on Instance uuid c7be44ac-3b42-4ace-ac56-65c117e839c6 {{(pid=118962) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 30 01:05:52.512025 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-21592fe7-3e50-4afa-bb13-daac2db58218 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:05:52.523881 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-21592fe7-3e50-4afa-bb13-daac2db58218 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6" status: 200 len: 1312 microversion: 2.1 time: 0.162575 Oct 30 01:05:52.524667 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 528/1056] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:05:52 2022] GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6 => generated 1312 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:05:53.165094 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-ff1dc68d-a0e3-47f1-b013-c3c2115a72a4 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:05:53.166099 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-ff1dc68d-a0e3-47f1-b013-c3c2115a72a4 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: b77a1841-4f87-4b8a-96a9-a619ea67d452] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:05:53.177167 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ff1dc68d-a0e3-47f1-b013-c3c2115a72a4 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:53.177980 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ff1dc68d-a0e3-47f1-b013-c3c2115a72a4 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:53.178719 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ff1dc68d-a0e3-47f1-b013-c3c2115a72a4 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:53.179913 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ff1dc68d-a0e3-47f1-b013-c3c2115a72a4 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:53.180672 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ff1dc68d-a0e3-47f1-b013-c3c2115a72a4 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:53.181378 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ff1dc68d-a0e3-47f1-b013-c3c2115a72a4 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:53.325811 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-ff1dc68d-a0e3-47f1-b013-c3c2115a72a4 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:05:53.337806 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-ff1dc68d-a0e3-47f1-b013-c3c2115a72a4 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452" status: 200 len: 1681 microversion: 2.1 time: 0.179414 Oct 30 01:05:53.338911 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 529/1057] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:05:53 2022] GET /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452 => generated 1681 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:05:53.550798 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-07a0c542-30d4-4c96-ab57-9dd8d9368f14 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:05:53.551400 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-07a0c542-30d4-4c96-ab57-9dd8d9368f14 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: c7be44ac-3b42-4ace-ac56-65c117e839c6] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:05:53.558278 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-07a0c542-30d4-4c96-ab57-9dd8d9368f14 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:53.559504 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-07a0c542-30d4-4c96-ab57-9dd8d9368f14 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:53.560106 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-07a0c542-30d4-4c96-ab57-9dd8d9368f14 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:53.561009 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-07a0c542-30d4-4c96-ab57-9dd8d9368f14 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:53.561545 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-07a0c542-30d4-4c96-ab57-9dd8d9368f14 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:53.562057 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-07a0c542-30d4-4c96-ab57-9dd8d9368f14 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:53.598072 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-07a0c542-30d4-4c96-ab57-9dd8d9368f14 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:53.598550 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-07a0c542-30d4-4c96-ab57-9dd8d9368f14 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:53.598983 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-07a0c542-30d4-4c96-ab57-9dd8d9368f14 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:53.599510 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.objects.instance [None req-07a0c542-30d4-4c96-ab57-9dd8d9368f14 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lazy-loading 'fault' on Instance uuid c7be44ac-3b42-4ace-ac56-65c117e839c6 {{(pid=118962) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 30 01:05:53.645444 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-07a0c542-30d4-4c96-ab57-9dd8d9368f14 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '31b7342ecf07408cab6bb040f361fa1f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48276ac51dd4bb1912fc670433cf6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:05:53.653353 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-07a0c542-30d4-4c96-ab57-9dd8d9368f14 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6" status: 200 len: 1312 microversion: 2.1 time: 0.109033 Oct 30 01:05:53.653916 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 529/1058] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:05:53 2022] GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6 => generated 1312 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:05:54.362688 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-1d519c8f-ae36-4bbe-a62f-e7be93ec9d35 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:05:54.363285 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-1d519c8f-ae36-4bbe-a62f-e7be93ec9d35 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: b77a1841-4f87-4b8a-96a9-a619ea67d452] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:05:54.369932 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1d519c8f-ae36-4bbe-a62f-e7be93ec9d35 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:54.370433 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1d519c8f-ae36-4bbe-a62f-e7be93ec9d35 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:54.370859 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1d519c8f-ae36-4bbe-a62f-e7be93ec9d35 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:54.371551 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1d519c8f-ae36-4bbe-a62f-e7be93ec9d35 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:54.372025 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1d519c8f-ae36-4bbe-a62f-e7be93ec9d35 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:54.372441 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1d519c8f-ae36-4bbe-a62f-e7be93ec9d35 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:54.514115 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-1d519c8f-ae36-4bbe-a62f-e7be93ec9d35 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:05:54.522318 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-1d519c8f-ae36-4bbe-a62f-e7be93ec9d35 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452" status: 200 len: 1681 microversion: 2.1 time: 0.163567 Oct 30 01:05:54.522895 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 530/1059] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:05:54 2022] GET /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452 => generated 1681 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:05:54.677086 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-bcc3f04b-c219-43e6-88de-db50303149d4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:05:54.677717 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-bcc3f04b-c219-43e6-88de-db50303149d4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] [instance: c7be44ac-3b42-4ace-ac56-65c117e839c6] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:05:54.684528 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bcc3f04b-c219-43e6-88de-db50303149d4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:54.685013 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bcc3f04b-c219-43e6-88de-db50303149d4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:54.685451 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bcc3f04b-c219-43e6-88de-db50303149d4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:54.686146 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bcc3f04b-c219-43e6-88de-db50303149d4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:54.686598 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bcc3f04b-c219-43e6-88de-db50303149d4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:54.686999 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bcc3f04b-c219-43e6-88de-db50303149d4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:54.700911 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.wsgi [None req-bcc3f04b-c219-43e6-88de-db50303149d4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] HTTP exception thrown: Instance c7be44ac-3b42-4ace-ac56-65c117e839c6 could not be found. Oct 30 01:05:54.702591 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-bcc3f04b-c219-43e6-88de-db50303149d4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Request method failure captured: Oct 30 01:05:54.702591 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: request: GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6 HTTP/1.1 Oct 30 01:05:54.702591 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept: application/json Oct 30 01:05:54.702591 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept-Encoding: identity Oct 30 01:05:54.702591 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Connection: close Oct 30 01:05:54.702591 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Length: 0 Oct 30 01:05:54.702591 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Type: application/json Oct 30 01:05:54.702591 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Host: 10.222.0.16 Oct 30 01:05:54.702591 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Openstack-System-Scope: None Oct 30 01:05:54.702591 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: User-Agent: python-urllib3/1.26.12 Oct 30 01:05:54.702591 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Auth-Token: gAAAAABjXcwtPy0eLS3MC3-cmfmCjFSfeavovRIYXfEfNVAywVbIQ30ctEe5V4sGvi1A3M4sDeUVrGvF9i6stAWwO3TejIedmS8RLhBtx-h_JsINfLnyH884rCMd4-euC7isoMsjNUbUBUpUbdHwx6JXx5oDJcwXMr1RZh9X14AJM7k_e6z6bu8 Oct 30 01:05:54.702591 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Id: None Oct 30 01:05:54.702591 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Name: None Oct 30 01:05:54.702591 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Identity-Status: Confirmed Oct 30 01:05:54.702591 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Is-Admin-Project: True Oct 30 01:05:54.702591 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Id: default Oct 30 01:05:54.702591 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Name: Default Oct 30 01:05:54.702591 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Id: f48276ac51dd4bb1912fc670433cf6c4 Oct 30 01:05:54.702591 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-764720217 Oct 30 01:05:54.702591 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Role: reader,member Oct 30 01:05:54.702591 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Roles: reader,member Oct 30 01:05:54.704116 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_f48276ac51dd4bb1912fc670433cf6c4"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f48276ac51dd4bb1912fc670433cf6c4"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/f48276ac51dd4bb1912fc670433cf6c4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f48276ac51dd4bb1912fc670433cf6c4"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 01:05:54.704116 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant: tempest-ServerRescueNegativeTestJSON-764720217 Oct 30 01:05:54.704116 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Id: f48276ac51dd4bb1912fc670433cf6c4 Oct 30 01:05:54.704116 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-764720217 Oct 30 01:05:54.704116 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User: tempest-ServerRescueNegativeTestJSON-764720217-project Oct 30 01:05:54.704116 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Id: default Oct 30 01:05:54.704116 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Name: Default Oct 30 01:05:54.704116 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Id: 31b7342ecf07408cab6bb040f361fa1f Oct 30 01:05:54.704116 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Name: tempest-ServerRescueNegativeTestJSON-764720217-project Oct 30 01:05:54.704116 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: method: > Oct 30 01:05:54.704116 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: exception: Instance c7be44ac-3b42-4ace-ac56-65c117e839c6 could not be found. Oct 30 01:05:54.704116 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: action_args: {'id': 'c7be44ac-3b42-4ace-ac56-65c117e839c6'} Oct 30 01:05:54.704905 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 01:05:54.704905 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:05:54.704905 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 30 01:05:54.704905 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 30 01:05:54.704905 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 30 01:05:54.704905 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 30 01:05:54.704905 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 30 01:05:54.704905 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 30 01:05:54.704905 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 30 01:05:54.704905 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 30 01:05:54.704905 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c7be44ac-3b42-4ace-ac56-65c117e839c6 could not be found. Oct 30 01:05:54.704905 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:05:54.704905 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:05:54.704905 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:05:54.704905 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:05:54.704905 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:05:54.704905 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:05:54.704905 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 01:05:54.704905 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 01:05:54.704905 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 01:05:54.706212 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 01:05:54.706212 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 30 01:05:54.706212 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 30 01:05:54.706212 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 30 01:05:54.706212 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 30 01:05:54.706212 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 30 01:05:54.706212 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 30 01:05:54.706212 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c7be44ac-3b42-4ace-ac56-65c117e839c6 could not be found. Oct 30 01:05:54.706212 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:05:54.706212 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:05:54.706212 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:05:54.706212 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:05:54.706212 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:05:54.706212 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:05:54.706212 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 01:05:54.706212 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 01:05:54.706212 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c7be44ac-3b42-4ace-ac56-65c117e839c6 could not be found. Oct 30 01:05:54.706212 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:05:54.707351 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-bcc3f04b-c219-43e6-88de-db50303149d4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] Returning 404 to user: Instance c7be44ac-3b42-4ace-ac56-65c117e839c6 could not be found. {{(pid=118962) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 01:05:54.707351 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-bcc3f04b-c219-43e6-88de-db50303149d4 tempest-ServerRescueNegativeTestJSON-764720217 tempest-ServerRescueNegativeTestJSON-764720217-project] 10.222.0.16 "GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6" status: 404 len: 111 microversion: 2.1 time: 0.030884 Oct 30 01:05:54.707351 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 530/1060] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:05:54 2022] GET /compute/v2.1/servers/c7be44ac-3b42-4ace-ac56-65c117e839c6 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 30 01:05:55.286671 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-163808d3-ec99-47f4-b7f6-839cd1fee1fb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:05:55.287590 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-163808d3-ec99-47f4-b7f6-839cd1fee1fb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:05:55.294006 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-163808d3-ec99-47f4-b7f6-839cd1fee1fb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:55.294790 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-163808d3-ec99-47f4-b7f6-839cd1fee1fb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:55.295454 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-163808d3-ec99-47f4-b7f6-839cd1fee1fb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:55.296655 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-163808d3-ec99-47f4-b7f6-839cd1fee1fb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:55.297332 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-163808d3-ec99-47f4-b7f6-839cd1fee1fb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:55.297838 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-163808d3-ec99-47f4-b7f6-839cd1fee1fb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:55.317281 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [req-b93ff866-d3f4-483a-8958-55772b5cb8fd req-1ad5fdca-16a5-47da-957e-3368329677b9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b77a1841-4f87-4b8a-96a9-a619ea67d452", "name": "network-vif-deleted", "tag": "a89f7199-cd3e-44d3-af70-4235cf4ab1a1"}]} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:05:55.332828 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-b93ff866-d3f4-483a-8958-55772b5cb8fd req-1ad5fdca-16a5-47da-957e-3368329677b9 service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:55.333828 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-b93ff866-d3f4-483a-8958-55772b5cb8fd req-1ad5fdca-16a5-47da-957e-3368329677b9 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:55.334508 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-163808d3-ec99-47f4-b7f6-839cd1fee1fb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20/ips" status: 200 len: 152 microversion: 2.20 time: 0.051970 Oct 30 01:05:55.335505 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 531/1061] 10.222.0.16 () {60 vars in 1289 bytes} [Sun Oct 30 01:05:55 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20/ips => generated 152 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 30 01:05:55.336098 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-b93ff866-d3f4-483a-8958-55772b5cb8fd req-1ad5fdca-16a5-47da-957e-3368329677b9 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:55.372942 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.compute.server_external_events [req-b93ff866-d3f4-483a-8958-55772b5cb8fd req-1ad5fdca-16a5-47da-957e-3368329677b9 service nova] Creating event network-vif-deleted:a89f7199-cd3e-44d3-af70-4235cf4ab1a1 for instance b77a1841-4f87-4b8a-96a9-a619ea67d452 on hv-cinder-maste Oct 30 01:05:55.384583 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [req-b93ff866-d3f4-483a-8958-55772b5cb8fd req-1ad5fdca-16a5-47da-957e-3368329677b9 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.071502 Oct 30 01:05:55.389807 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 531/1062] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:05:55 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:05:55.550580 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-83d335da-0d63-4ee9-9c89-59fe0bf858b0 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:05:55.551818 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-83d335da-0d63-4ee9-9c89-59fe0bf858b0 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: b77a1841-4f87-4b8a-96a9-a619ea67d452] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:05:55.565698 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-83d335da-0d63-4ee9-9c89-59fe0bf858b0 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:55.566195 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-83d335da-0d63-4ee9-9c89-59fe0bf858b0 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:55.566634 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-83d335da-0d63-4ee9-9c89-59fe0bf858b0 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:55.569854 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-83d335da-0d63-4ee9-9c89-59fe0bf858b0 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:55.570547 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-83d335da-0d63-4ee9-9c89-59fe0bf858b0 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:55.570994 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-83d335da-0d63-4ee9-9c89-59fe0bf858b0 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:55.654316 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-83d335da-0d63-4ee9-9c89-59fe0bf858b0 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:55.655114 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-83d335da-0d63-4ee9-9c89-59fe0bf858b0 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:55.655978 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-83d335da-0d63-4ee9-9c89-59fe0bf858b0 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:55.657964 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.objects.instance [None req-83d335da-0d63-4ee9-9c89-59fe0bf858b0 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lazy-loading 'fault' on Instance uuid b77a1841-4f87-4b8a-96a9-a619ea67d452 {{(pid=118961) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 30 01:05:55.779392 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-83d335da-0d63-4ee9-9c89-59fe0bf858b0 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:05:55.787536 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-83d335da-0d63-4ee9-9c89-59fe0bf858b0 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452" status: 200 len: 1331 microversion: 2.1 time: 0.245662 Oct 30 01:05:55.788400 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 532/1063] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:05:55 2022] GET /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452 => generated 1331 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:05:56.812990 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-6ad4c742-162a-4bb6-b8d8-03333f831742 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:05:56.813824 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-6ad4c742-162a-4bb6-b8d8-03333f831742 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: b77a1841-4f87-4b8a-96a9-a619ea67d452] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:05:56.824802 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6ad4c742-162a-4bb6-b8d8-03333f831742 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:56.827124 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6ad4c742-162a-4bb6-b8d8-03333f831742 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:56.827666 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6ad4c742-162a-4bb6-b8d8-03333f831742 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:56.830447 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6ad4c742-162a-4bb6-b8d8-03333f831742 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:56.831158 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6ad4c742-162a-4bb6-b8d8-03333f831742 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:56.831845 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6ad4c742-162a-4bb6-b8d8-03333f831742 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:56.860432 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.wsgi [None req-6ad4c742-162a-4bb6-b8d8-03333f831742 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] HTTP exception thrown: Instance b77a1841-4f87-4b8a-96a9-a619ea67d452 could not be found. Oct 30 01:05:56.873612 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-6ad4c742-162a-4bb6-b8d8-03333f831742 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Request method failure captured: Oct 30 01:05:56.873612 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: request: GET /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452 HTTP/1.1 Oct 30 01:05:56.873612 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept: application/json Oct 30 01:05:56.873612 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept-Encoding: identity Oct 30 01:05:56.873612 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Connection: close Oct 30 01:05:56.873612 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Length: 0 Oct 30 01:05:56.873612 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Type: application/json Oct 30 01:05:56.873612 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Host: 10.222.0.16 Oct 30 01:05:56.873612 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Openstack-System-Scope: None Oct 30 01:05:56.873612 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: User-Agent: python-urllib3/1.26.12 Oct 30 01:05:56.873612 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Auth-Token: gAAAAABjXcuedsaRjtnPRcHo4HqTnifOYe1vcHHjSzl7t8SsEKfOt41JBob3t9Ien606ucEZmcgmpf5H0lzlL6oZS4vLpZERLur9y2YRhqd9AYPyhbrmUkrYmKAssv6PSCysf6z3-myn1YKr62Nxqfs7t1lZGZnZuH_I2j8LHl5PnVXWJxyADCY Oct 30 01:05:56.873612 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Id: None Oct 30 01:05:56.873612 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Name: None Oct 30 01:05:56.873612 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Identity-Status: Confirmed Oct 30 01:05:56.873612 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Is-Admin-Project: True Oct 30 01:05:56.873612 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Id: default Oct 30 01:05:56.873612 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Name: Default Oct 30 01:05:56.873612 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Id: 6b68b886c19c45d585409651f33b8626 Oct 30 01:05:56.873612 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Name: tempest-AttachVolumeNegativeTest-618009204 Oct 30 01:05:56.873612 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Role: reader,member Oct 30 01:05:56.873612 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Roles: reader,member Oct 30 01:05:56.875327 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_6b68b886c19c45d585409651f33b8626"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6b68b886c19c45d585409651f33b8626"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/6b68b886c19c45d585409651f33b8626"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6b68b886c19c45d585409651f33b8626"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 01:05:56.875327 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant: tempest-AttachVolumeNegativeTest-618009204 Oct 30 01:05:56.875327 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Id: 6b68b886c19c45d585409651f33b8626 Oct 30 01:05:56.875327 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-618009204 Oct 30 01:05:56.875327 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User: tempest-AttachVolumeNegativeTest-618009204-project Oct 30 01:05:56.875327 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Id: default Oct 30 01:05:56.875327 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Name: Default Oct 30 01:05:56.875327 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Id: ecaf8ec105c04b919256f852da859eba Oct 30 01:05:56.875327 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Name: tempest-AttachVolumeNegativeTest-618009204-project Oct 30 01:05:56.875327 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: method: > Oct 30 01:05:56.875327 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: exception: Instance b77a1841-4f87-4b8a-96a9-a619ea67d452 could not be found. Oct 30 01:05:56.875327 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: action_args: {'id': 'b77a1841-4f87-4b8a-96a9-a619ea67d452'} Oct 30 01:05:56.876746 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 01:05:56.876746 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:05:56.876746 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 30 01:05:56.876746 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 30 01:05:56.876746 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 30 01:05:56.876746 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 30 01:05:56.876746 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 30 01:05:56.876746 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 30 01:05:56.876746 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 30 01:05:56.876746 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 30 01:05:56.876746 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b77a1841-4f87-4b8a-96a9-a619ea67d452 could not be found. Oct 30 01:05:56.876746 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:05:56.876746 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:05:56.876746 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:05:56.876746 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:05:56.876746 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:05:56.876746 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:05:56.876746 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 01:05:56.876746 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 01:05:56.876746 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 01:05:56.882046 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 01:05:56.882046 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 30 01:05:56.882046 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 30 01:05:56.882046 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 30 01:05:56.882046 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 30 01:05:56.882046 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 30 01:05:56.882046 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 30 01:05:56.882046 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b77a1841-4f87-4b8a-96a9-a619ea67d452 could not be found. Oct 30 01:05:56.882046 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:05:56.882046 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:05:56.882046 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:05:56.882046 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:05:56.882046 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:05:56.882046 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:05:56.882046 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 01:05:56.882046 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 01:05:56.882046 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b77a1841-4f87-4b8a-96a9-a619ea67d452 could not be found. Oct 30 01:05:56.882046 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:05:56.884419 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-6ad4c742-162a-4bb6-b8d8-03333f831742 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Returning 404 to user: Instance b77a1841-4f87-4b8a-96a9-a619ea67d452 could not be found. {{(pid=118962) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 01:05:56.884419 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-6ad4c742-162a-4bb6-b8d8-03333f831742 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452" status: 404 len: 111 microversion: 2.1 time: 0.068559 Oct 30 01:05:56.884419 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 532/1064] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:05:56 2022] GET /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452 => generated 111 bytes in 70 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 30 01:05:56.887099 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-ce57cf0a-ca58-48f6-ac2a-bb71c9fde1a8 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '.version_select at 0x7f0eabacf790>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:05:56.916936 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-ce57cf0a-ca58-48f6-ac2a-bb71c9fde1a8 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-165130656" status: 202 len: 0 microversion: 2.1 time: 0.034358 Oct 30 01:05:56.921777 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 533/1065] 10.222.0.16 () {58 vars in 1218 bytes} [Sun Oct 30 01:05:56 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-165130656 => generated 0 bytes in 37 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 30 01:05:57.720026 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-351a719d-c6bb-4a9f-99f0-a5e3abe156c4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:05:57.720889 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-351a719d-c6bb-4a9f-99f0-a5e3abe156c4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:05:57.728501 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-351a719d-c6bb-4a9f-99f0-a5e3abe156c4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:57.729382 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-351a719d-c6bb-4a9f-99f0-a5e3abe156c4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:57.729882 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-351a719d-c6bb-4a9f-99f0-a5e3abe156c4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:57.730658 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-351a719d-c6bb-4a9f-99f0-a5e3abe156c4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:57.731172 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-351a719d-c6bb-4a9f-99f0-a5e3abe156c4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:57.731851 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-351a719d-c6bb-4a9f-99f0-a5e3abe156c4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:57.756787 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.objects.instance [None req-351a719d-c6bb-4a9f-99f0-a5e3abe156c4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lazy-loading 'flavor' on Instance uuid f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 {{(pid=118962) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 30 01:05:57.945759 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-5419ca8d-4e50-459a-9381-3227d2df56cc tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Action: 'create', calling method: .version_select at 0x7f0eabdeab80>, body: {"keypair": {"name": "tempest-keypair-741020447", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBARhvJla0d5O8lFRe6iy8PuaF8a/j1IKDWfBjFGP83Kx7Ere6Kv3KdXU1KQjrzraEfnDM5uzlcTj2qFshzANppBE12oG6mSQpn3hPLGFYCYJgNJttYFD2g8gei4aWhEc/w=="}} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:05:57.969239 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.quota [None req-5419ca8d-4e50-459a-9381-3227d2df56cc tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Getting quotas for project 6b68b886c19c45d585409651f33b8626. Resources: {'key_pairs'} {{(pid=118961) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 30 01:05:57.973578 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.quota [None req-5419ca8d-4e50-459a-9381-3227d2df56cc tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Getting quotas for user ecaf8ec105c04b919256f852da859eba and project 6b68b886c19c45d585409651f33b8626. Resources: {'key_pairs'} {{(pid=118961) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 30 01:05:58.004379 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-5419ca8d-4e50-459a-9381-3227d2df56cc tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.064439 Oct 30 01:05:58.009626 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-351a719d-c6bb-4a9f-99f0-a5e3abe156c4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "POST /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20/action" status: 202 len: 0 microversion: 2.20 time: 0.294784 Oct 30 01:05:58.010625 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 533/1066] 10.222.0.16 () {62 vars in 1319 bytes} [Sun Oct 30 01:05:57 2022] POST /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20/action => generated 0 bytes in 297 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 30 01:05:58.014607 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 534/1067] 10.222.0.16 () {60 vars in 1159 bytes} [Sun Oct 30 01:05:57 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:05:58.020386 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-91cfd291-a304-46ac-b0cb-c5c4371bc53b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:05:58.022010 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-91cfd291-a304-46ac-b0cb-c5c4371bc53b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:05:58.031200 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-91cfd291-a304-46ac-b0cb-c5c4371bc53b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:58.033651 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-91cfd291-a304-46ac-b0cb-c5c4371bc53b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:58.034121 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-91cfd291-a304-46ac-b0cb-c5c4371bc53b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:58.035079 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-91cfd291-a304-46ac-b0cb-c5c4371bc53b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:58.035697 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-91cfd291-a304-46ac-b0cb-c5c4371bc53b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:58.037544 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-91cfd291-a304-46ac-b0cb-c5c4371bc53b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:58.337451 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-91cfd291-a304-46ac-b0cb-c5c4371bc53b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:05:58.350093 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-91cfd291-a304-46ac-b0cb-c5c4371bc53b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:05:58.352954 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-91cfd291-a304-46ac-b0cb-c5c4371bc53b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:05:58.354713 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-91cfd291-a304-46ac-b0cb-c5c4371bc53b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1723 microversion: 2.20 time: 0.339351 Oct 30 01:05:58.375487 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 534/1068] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:05:58 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1723 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:05:59.383635 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-6042a9f4-14fa-4f0e-8d41-0b7e84a6ce20 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:05:59.384672 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-6042a9f4-14fa-4f0e-8d41-0b7e84a6ce20 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:05:59.393547 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6042a9f4-14fa-4f0e-8d41-0b7e84a6ce20 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:59.394487 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6042a9f4-14fa-4f0e-8d41-0b7e84a6ce20 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:59.395012 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6042a9f4-14fa-4f0e-8d41-0b7e84a6ce20 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:59.396414 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6042a9f4-14fa-4f0e-8d41-0b7e84a6ce20 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:05:59.396971 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6042a9f4-14fa-4f0e-8d41-0b7e84a6ce20 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:05:59.397489 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6042a9f4-14fa-4f0e-8d41-0b7e84a6ce20 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:05:59.542706 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-6042a9f4-14fa-4f0e-8d41-0b7e84a6ce20 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:05:59.554910 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-6042a9f4-14fa-4f0e-8d41-0b7e84a6ce20 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:05:59.554910 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-6042a9f4-14fa-4f0e-8d41-0b7e84a6ce20 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:05:59.554910 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-6042a9f4-14fa-4f0e-8d41-0b7e84a6ce20 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1723 microversion: 2.20 time: 0.175881 Oct 30 01:05:59.555285 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 535/1069] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:05:59 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1723 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:05:59.698678 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-ce47fdab-3ee0-4878-a14d-6942c5649177 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1183495855", "imageRef": "4c16f4f6-84cb-4463-b82c-b7c65f3826f0", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "4400f15d-ace0-4893-b5ff-c43f47dad5a5"}], "security_groups": [{"name": "tempest-securitygroup--1222024719"}], "key_name": "tempest-keypair-741020447", "user_data": "}} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:05:59.813546 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.network.neutron [None req-ce47fdab-3ee0-4878-a14d-6942c5649177 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] validate_networks() for [('4400f15d-ace0-4893-b5ff-c43f47dad5a5', None, None, None, None, None)] {{(pid=118962) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 30 01:06:00.305487 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.virt.hardware [None req-ce47fdab-3ee0-4878-a14d-6942c5649177 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Flavor limits 0:0:0 {{(pid=118962) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 30 01:06:00.305940 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.virt.hardware [None req-ce47fdab-3ee0-4878-a14d-6942c5649177 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Image limits 0:0:0 {{(pid=118962) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 30 01:06:00.306417 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.virt.hardware [None req-ce47fdab-3ee0-4878-a14d-6942c5649177 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Flavor pref 0:0:0 {{(pid=118962) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 30 01:06:00.306820 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.virt.hardware [None req-ce47fdab-3ee0-4878-a14d-6942c5649177 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Image pref 0:0:0 {{(pid=118962) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 30 01:06:00.307224 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.virt.hardware [None req-ce47fdab-3ee0-4878-a14d-6942c5649177 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118962) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 30 01:06:00.316121 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ce47fdab-3ee0-4878-a14d-6942c5649177 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:00.316606 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ce47fdab-3ee0-4878-a14d-6942c5649177 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:00.317040 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ce47fdab-3ee0-4878-a14d-6942c5649177 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:00.317847 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ce47fdab-3ee0-4878-a14d-6942c5649177 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:00.318423 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ce47fdab-3ee0-4878-a14d-6942c5649177 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:00.318839 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ce47fdab-3ee0-4878-a14d-6942c5649177 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:00.335631 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.quota [None req-ce47fdab-3ee0-4878-a14d-6942c5649177 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Getting quotas for project 6b68b886c19c45d585409651f33b8626. Resources: {'ram', 'instances', 'cores'} {{(pid=118962) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 30 01:06:00.339940 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.quota [None req-ce47fdab-3ee0-4878-a14d-6942c5649177 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Getting quotas for user ecaf8ec105c04b919256f852da859eba and project 6b68b886c19c45d585409651f33b8626. Resources: {'ram', 'instances', 'cores'} {{(pid=118962) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 30 01:06:00.351243 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-ce47fdab-3ee0-4878-a14d-6942c5649177 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Going to run 1 instances... {{(pid=118962) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 30 01:06:00.353616 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-ce47fdab-3ee0-4878-a14d-6942c5649177 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: ffdd5acc-f9cf-4244-ae3b-429260b7c33f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4c16f4f6-84cb-4463-b82c-b7c65f3826f0',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118962) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 30 01:06:00.399703 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR oslo.messaging._drivers.impl_rabbit [None req-ce47fdab-3ee0-4878-a14d-6942c5649177 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [d477ea54-f059-4108-a6bc-754d6834b6d1] AMQP server on 10.222.0.16:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Oct 30 01:06:00.580360 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-c3227bcf-2066-4a30-99c4-08b1c02e96b1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:00.581008 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-c3227bcf-2066-4a30-99c4-08b1c02e96b1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:00.587761 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c3227bcf-2066-4a30-99c4-08b1c02e96b1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:00.588287 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c3227bcf-2066-4a30-99c4-08b1c02e96b1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:00.589043 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c3227bcf-2066-4a30-99c4-08b1c02e96b1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:00.589740 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c3227bcf-2066-4a30-99c4-08b1c02e96b1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:00.590187 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c3227bcf-2066-4a30-99c4-08b1c02e96b1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:00.590605 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c3227bcf-2066-4a30-99c4-08b1c02e96b1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:00.722487 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-c3227bcf-2066-4a30-99c4-08b1c02e96b1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:00.730884 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-c3227bcf-2066-4a30-99c4-08b1c02e96b1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:00.732514 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-c3227bcf-2066-4a30-99c4-08b1c02e96b1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:00.733686 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-c3227bcf-2066-4a30-99c4-08b1c02e96b1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1723 microversion: 2.20 time: 0.157443 Oct 30 01:06:00.734247 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 536/1070] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:06:00 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1723 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:06:01.407527 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO oslo.messaging._drivers.impl_rabbit [None req-ce47fdab-3ee0-4878-a14d-6942c5649177 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [d477ea54-f059-4108-a6bc-754d6834b6d1] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 59256. Oct 30 01:06:01.412048 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-ce47fdab-3ee0-4878-a14d-6942c5649177 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 1.717837 Oct 30 01:06:01.412755 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 535/1071] 10.222.0.16 () {60 vars in 1147 bytes} [Sun Oct 30 01:05:59 2022] POST /compute/v2.1/servers => generated 402 bytes in 1719 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 30 01:06:01.693266 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-367eced4-a8ad-48e1-b93f-80b3c6c5cb50 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:01.693884 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-367eced4-a8ad-48e1-b93f-80b3c6c5cb50 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: ffdd5acc-f9cf-4244-ae3b-429260b7c33f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:01.751907 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-367eced4-a8ad-48e1-b93f-80b3c6c5cb50 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:01.760038 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-367eced4-a8ad-48e1-b93f-80b3c6c5cb50 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/ffdd5acc-f9cf-4244-ae3b-429260b7c33f" status: 200 len: 1241 microversion: 2.1 time: 0.339759 Oct 30 01:06:01.760939 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 537/1072] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:06:01 2022] GET /compute/v2.1/servers/ffdd5acc-f9cf-4244-ae3b-429260b7c33f => generated 1241 bytes in 341 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:06:02.020573 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-4de39024-1de7-4cd3-aeda-6764cfb4c8c5 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:02.021322 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-4de39024-1de7-4cd3-aeda-6764cfb4c8c5 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:02.031008 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4de39024-1de7-4cd3-aeda-6764cfb4c8c5 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:02.032438 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4de39024-1de7-4cd3-aeda-6764cfb4c8c5 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:02.032880 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4de39024-1de7-4cd3-aeda-6764cfb4c8c5 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:02.033586 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4de39024-1de7-4cd3-aeda-6764cfb4c8c5 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:02.034040 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4de39024-1de7-4cd3-aeda-6764cfb4c8c5 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:02.034454 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4de39024-1de7-4cd3-aeda-6764cfb4c8c5 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:02.187442 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-4de39024-1de7-4cd3-aeda-6764cfb4c8c5 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:02.196985 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-4de39024-1de7-4cd3-aeda-6764cfb4c8c5 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:02.199136 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-4de39024-1de7-4cd3-aeda-6764cfb4c8c5 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:02.200419 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-4de39024-1de7-4cd3-aeda-6764cfb4c8c5 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1723 microversion: 2.20 time: 0.445463 Oct 30 01:06:02.201141 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 536/1073] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:06:01 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1723 bytes in 447 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:06:02.784141 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-cc4adb37-8fc3-40ae-9924-8cc4c27efce8 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:02.785532 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-cc4adb37-8fc3-40ae-9924-8cc4c27efce8 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: ffdd5acc-f9cf-4244-ae3b-429260b7c33f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:02.793241 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cc4adb37-8fc3-40ae-9924-8cc4c27efce8 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:02.794261 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cc4adb37-8fc3-40ae-9924-8cc4c27efce8 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:02.794949 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cc4adb37-8fc3-40ae-9924-8cc4c27efce8 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:02.796271 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cc4adb37-8fc3-40ae-9924-8cc4c27efce8 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:02.796896 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cc4adb37-8fc3-40ae-9924-8cc4c27efce8 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:02.797519 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cc4adb37-8fc3-40ae-9924-8cc4c27efce8 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:02.876395 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-cc4adb37-8fc3-40ae-9924-8cc4c27efce8 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:02.885643 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-cc4adb37-8fc3-40ae-9924-8cc4c27efce8 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/ffdd5acc-f9cf-4244-ae3b-429260b7c33f" status: 200 len: 1311 microversion: 2.1 time: 0.106335 Oct 30 01:06:02.886231 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 538/1074] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:06:02 2022] GET /compute/v2.1/servers/ffdd5acc-f9cf-4244-ae3b-429260b7c33f => generated 1311 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:06:03.226335 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-88e12073-f246-4b72-8408-824df2fbab85 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:03.226945 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-88e12073-f246-4b72-8408-824df2fbab85 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:03.234076 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-88e12073-f246-4b72-8408-824df2fbab85 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:03.234666 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-88e12073-f246-4b72-8408-824df2fbab85 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:03.235170 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-88e12073-f246-4b72-8408-824df2fbab85 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:03.235972 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-88e12073-f246-4b72-8408-824df2fbab85 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:03.236430 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-88e12073-f246-4b72-8408-824df2fbab85 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:03.236886 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-88e12073-f246-4b72-8408-824df2fbab85 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:03.388973 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-88e12073-f246-4b72-8408-824df2fbab85 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:03.400579 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-88e12073-f246-4b72-8408-824df2fbab85 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:03.403393 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-88e12073-f246-4b72-8408-824df2fbab85 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:03.404858 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-88e12073-f246-4b72-8408-824df2fbab85 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1723 microversion: 2.20 time: 0.182799 Oct 30 01:06:03.407547 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 537/1075] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:06:03 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1723 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:06:03.912552 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-57859f3c-b0cf-4a9c-99d7-319c76a09b6c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:03.913802 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-57859f3c-b0cf-4a9c-99d7-319c76a09b6c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: ffdd5acc-f9cf-4244-ae3b-429260b7c33f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:03.933247 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_api:SELECT=81,nova_api:UPDATE=2,nova_api:DELETE=1,nova_api:INSERT=1 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:06:03.934677 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-57859f3c-b0cf-4a9c-99d7-319c76a09b6c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:03.935965 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-57859f3c-b0cf-4a9c-99d7-319c76a09b6c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:03.936409 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-57859f3c-b0cf-4a9c-99d7-319c76a09b6c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:03.937164 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-57859f3c-b0cf-4a9c-99d7-319c76a09b6c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:03.937605 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-57859f3c-b0cf-4a9c-99d7-319c76a09b6c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:03.938095 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-57859f3c-b0cf-4a9c-99d7-319c76a09b6c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:04.083449 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-57859f3c-b0cf-4a9c-99d7-319c76a09b6c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:04.094194 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-57859f3c-b0cf-4a9c-99d7-319c76a09b6c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/ffdd5acc-f9cf-4244-ae3b-429260b7c33f" status: 200 len: 1367 microversion: 2.1 time: 0.188204 Oct 30 01:06:04.095404 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 539/1076] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:06:03 2022] GET /compute/v2.1/servers/ffdd5acc-f9cf-4244-ae3b-429260b7c33f => generated 1367 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:06:04.431949 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-7c03f6b5-9bf5-48e7-8593-bb218f4d07b4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:04.432550 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-7c03f6b5-9bf5-48e7-8593-bb218f4d07b4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:04.435000 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_api:SELECT=83,nova_api:DELETE=1,nova_api:UPDATE=2,nova_api:INSERT=3 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:06:04.440881 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7c03f6b5-9bf5-48e7-8593-bb218f4d07b4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:04.441366 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7c03f6b5-9bf5-48e7-8593-bb218f4d07b4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:04.441794 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7c03f6b5-9bf5-48e7-8593-bb218f4d07b4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:04.442677 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7c03f6b5-9bf5-48e7-8593-bb218f4d07b4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:04.443285 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7c03f6b5-9bf5-48e7-8593-bb218f4d07b4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:04.443699 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7c03f6b5-9bf5-48e7-8593-bb218f4d07b4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:04.577320 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-7c03f6b5-9bf5-48e7-8593-bb218f4d07b4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:04.586038 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-7c03f6b5-9bf5-48e7-8593-bb218f4d07b4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:04.587496 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-7c03f6b5-9bf5-48e7-8593-bb218f4d07b4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:04.589102 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-7c03f6b5-9bf5-48e7-8593-bb218f4d07b4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1723 microversion: 2.20 time: 0.160845 Oct 30 01:06:04.589663 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 538/1077] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:06:04 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1723 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:06:04.624426 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [req-799b6552-8c2e-4118-a9c1-510b9877d9d5 req-a78fc99b-a833-4ce8-8620-844faf91af4e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ffdd5acc-f9cf-4244-ae3b-429260b7c33f", "tag": "c36e47a4-5169-485f-88ec-96caef1c3d0f"}]} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:06:04.635017 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-799b6552-8c2e-4118-a9c1-510b9877d9d5 req-a78fc99b-a833-4ce8-8620-844faf91af4e service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:04.635807 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-799b6552-8c2e-4118-a9c1-510b9877d9d5 req-a78fc99b-a833-4ce8-8620-844faf91af4e service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:04.636249 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-799b6552-8c2e-4118-a9c1-510b9877d9d5 req-a78fc99b-a833-4ce8-8620-844faf91af4e service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:04.647905 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.compute.server_external_events [req-799b6552-8c2e-4118-a9c1-510b9877d9d5 req-a78fc99b-a833-4ce8-8620-844faf91af4e service nova] Creating event network-changed:c36e47a4-5169-485f-88ec-96caef1c3d0f for instance ffdd5acc-f9cf-4244-ae3b-429260b7c33f on hv-cinder-maste Oct 30 01:06:04.652888 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [req-799b6552-8c2e-4118-a9c1-510b9877d9d5 req-a78fc99b-a833-4ce8-8620-844faf91af4e service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032335 Oct 30 01:06:04.653509 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 540/1078] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:06:04 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:06:05.117661 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-d253f54f-ce28-4f75-9018-36b745ffc89c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:05.118798 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-d253f54f-ce28-4f75-9018-36b745ffc89c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: ffdd5acc-f9cf-4244-ae3b-429260b7c33f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:05.127252 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d253f54f-ce28-4f75-9018-36b745ffc89c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:05.128480 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d253f54f-ce28-4f75-9018-36b745ffc89c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:05.128905 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d253f54f-ce28-4f75-9018-36b745ffc89c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:05.129597 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d253f54f-ce28-4f75-9018-36b745ffc89c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:05.130320 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d253f54f-ce28-4f75-9018-36b745ffc89c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:05.130740 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d253f54f-ce28-4f75-9018-36b745ffc89c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:05.267326 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-d253f54f-ce28-4f75-9018-36b745ffc89c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:05.275199 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-d253f54f-ce28-4f75-9018-36b745ffc89c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/ffdd5acc-f9cf-4244-ae3b-429260b7c33f" status: 200 len: 1367 microversion: 2.1 time: 0.162204 Oct 30 01:06:05.275811 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 539/1079] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:06:05 2022] GET /compute/v2.1/servers/ffdd5acc-f9cf-4244-ae3b-429260b7c33f => generated 1367 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 01:06:05.618868 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-20380a24-180d-44d2-999c-2cd0b78956ed tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:05.619497 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-20380a24-180d-44d2-999c-2cd0b78956ed tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:05.626551 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-20380a24-180d-44d2-999c-2cd0b78956ed tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:05.627052 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-20380a24-180d-44d2-999c-2cd0b78956ed tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:05.627504 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-20380a24-180d-44d2-999c-2cd0b78956ed tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:05.628265 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-20380a24-180d-44d2-999c-2cd0b78956ed tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:05.628695 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-20380a24-180d-44d2-999c-2cd0b78956ed tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:05.629169 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-20380a24-180d-44d2-999c-2cd0b78956ed tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:05.768656 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-20380a24-180d-44d2-999c-2cd0b78956ed tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:05.777401 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-20380a24-180d-44d2-999c-2cd0b78956ed tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:05.778811 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-20380a24-180d-44d2-999c-2cd0b78956ed tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:05.779993 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-20380a24-180d-44d2-999c-2cd0b78956ed tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1723 microversion: 2.20 time: 0.167517 Oct 30 01:06:05.780555 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 541/1080] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:06:05 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1723 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:06:06.302172 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-99c69feb-a696-4940-a8ed-1444bad9081d tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:06.302873 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-99c69feb-a696-4940-a8ed-1444bad9081d tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: ffdd5acc-f9cf-4244-ae3b-429260b7c33f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:06.309828 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-99c69feb-a696-4940-a8ed-1444bad9081d tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:06.310321 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-99c69feb-a696-4940-a8ed-1444bad9081d tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:06.310760 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-99c69feb-a696-4940-a8ed-1444bad9081d tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:06.311456 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-99c69feb-a696-4940-a8ed-1444bad9081d tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:06.311955 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-99c69feb-a696-4940-a8ed-1444bad9081d tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:06.312367 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-99c69feb-a696-4940-a8ed-1444bad9081d tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:06.443373 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-99c69feb-a696-4940-a8ed-1444bad9081d tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:06.451174 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-99c69feb-a696-4940-a8ed-1444bad9081d tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/ffdd5acc-f9cf-4244-ae3b-429260b7c33f" status: 200 len: 1367 microversion: 2.1 time: 0.155207 Oct 30 01:06:06.451745 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 540/1081] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:06:06 2022] GET /compute/v2.1/servers/ffdd5acc-f9cf-4244-ae3b-429260b7c33f => generated 1367 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:06:06.805600 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-8dcb82e7-61ad-4eb9-8806-b37de5fe700f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:06.806133 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-8dcb82e7-61ad-4eb9-8806-b37de5fe700f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:06.812863 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8dcb82e7-61ad-4eb9-8806-b37de5fe700f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:06.813452 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8dcb82e7-61ad-4eb9-8806-b37de5fe700f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:06.813905 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8dcb82e7-61ad-4eb9-8806-b37de5fe700f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:06.814599 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8dcb82e7-61ad-4eb9-8806-b37de5fe700f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:06.815043 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8dcb82e7-61ad-4eb9-8806-b37de5fe700f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:06.815459 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8dcb82e7-61ad-4eb9-8806-b37de5fe700f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:06.956097 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-8dcb82e7-61ad-4eb9-8806-b37de5fe700f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:06.964945 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-8dcb82e7-61ad-4eb9-8806-b37de5fe700f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:06.966362 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-8dcb82e7-61ad-4eb9-8806-b37de5fe700f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:06.967518 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-8dcb82e7-61ad-4eb9-8806-b37de5fe700f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1723 microversion: 2.20 time: 0.166029 Oct 30 01:06:06.968096 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 542/1082] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:06:06 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1723 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:06:07.474201 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-02b46edc-6fdf-48c9-8457-daec5a48b7c1 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:07.475512 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-02b46edc-6fdf-48c9-8457-daec5a48b7c1 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: ffdd5acc-f9cf-4244-ae3b-429260b7c33f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:07.482274 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-02b46edc-6fdf-48c9-8457-daec5a48b7c1 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:07.482845 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-02b46edc-6fdf-48c9-8457-daec5a48b7c1 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:07.483260 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-02b46edc-6fdf-48c9-8457-daec5a48b7c1 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:07.484005 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-02b46edc-6fdf-48c9-8457-daec5a48b7c1 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:07.484447 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-02b46edc-6fdf-48c9-8457-daec5a48b7c1 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:07.484863 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-02b46edc-6fdf-48c9-8457-daec5a48b7c1 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:07.668014 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-02b46edc-6fdf-48c9-8457-daec5a48b7c1 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:07.676154 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-02b46edc-6fdf-48c9-8457-daec5a48b7c1 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/ffdd5acc-f9cf-4244-ae3b-429260b7c33f" status: 200 len: 1367 microversion: 2.1 time: 0.205925 Oct 30 01:06:07.676849 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 541/1083] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:06:07 2022] GET /compute/v2.1/servers/ffdd5acc-f9cf-4244-ae3b-429260b7c33f => generated 1367 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:06:07.993425 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-4d24db80-e1db-422d-807d-23c4f10c30c4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:07.994060 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-4d24db80-e1db-422d-807d-23c4f10c30c4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:08.001248 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4d24db80-e1db-422d-807d-23c4f10c30c4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:08.001772 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4d24db80-e1db-422d-807d-23c4f10c30c4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:08.002216 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4d24db80-e1db-422d-807d-23c4f10c30c4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:08.002934 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4d24db80-e1db-422d-807d-23c4f10c30c4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:08.003387 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4d24db80-e1db-422d-807d-23c4f10c30c4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:08.003856 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4d24db80-e1db-422d-807d-23c4f10c30c4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:08.182512 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-4d24db80-e1db-422d-807d-23c4f10c30c4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:08.191080 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-4d24db80-e1db-422d-807d-23c4f10c30c4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:08.192673 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-4d24db80-e1db-422d-807d-23c4f10c30c4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:08.193995 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-4d24db80-e1db-422d-807d-23c4f10c30c4 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1723 microversion: 2.20 time: 0.204685 Oct 30 01:06:08.194559 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 543/1084] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:06:07 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1723 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:06:08.700926 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-e71411f2-3825-4376-9830-d2f5f13c8908 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:08.702267 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-e71411f2-3825-4376-9830-d2f5f13c8908 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: ffdd5acc-f9cf-4244-ae3b-429260b7c33f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:08.709656 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e71411f2-3825-4376-9830-d2f5f13c8908 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:08.710603 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e71411f2-3825-4376-9830-d2f5f13c8908 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:08.711337 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e71411f2-3825-4376-9830-d2f5f13c8908 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:08.712298 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e71411f2-3825-4376-9830-d2f5f13c8908 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:08.712964 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e71411f2-3825-4376-9830-d2f5f13c8908 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:08.713645 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e71411f2-3825-4376-9830-d2f5f13c8908 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:08.850367 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-e71411f2-3825-4376-9830-d2f5f13c8908 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:08.858367 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-e71411f2-3825-4376-9830-d2f5f13c8908 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/ffdd5acc-f9cf-4244-ae3b-429260b7c33f" status: 200 len: 1367 microversion: 2.1 time: 0.163248 Oct 30 01:06:08.859185 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 542/1085] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:06:08 2022] GET /compute/v2.1/servers/ffdd5acc-f9cf-4244-ae3b-429260b7c33f => generated 1367 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:06:09.220257 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-37ef5c2d-cca4-4341-ac27-01fc40714835 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:09.220860 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-37ef5c2d-cca4-4341-ac27-01fc40714835 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:09.227451 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-37ef5c2d-cca4-4341-ac27-01fc40714835 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:09.227976 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-37ef5c2d-cca4-4341-ac27-01fc40714835 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:09.228479 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-37ef5c2d-cca4-4341-ac27-01fc40714835 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:09.229096 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-37ef5c2d-cca4-4341-ac27-01fc40714835 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:09.229542 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-37ef5c2d-cca4-4341-ac27-01fc40714835 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:09.229957 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-37ef5c2d-cca4-4341-ac27-01fc40714835 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:09.248805 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:06:09.369564 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-37ef5c2d-cca4-4341-ac27-01fc40714835 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:09.378559 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-37ef5c2d-cca4-4341-ac27-01fc40714835 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:09.380010 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-37ef5c2d-cca4-4341-ac27-01fc40714835 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:09.381174 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-37ef5c2d-cca4-4341-ac27-01fc40714835 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1723 microversion: 2.20 time: 0.164895 Oct 30 01:06:09.381724 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 544/1086] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:06:09 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1723 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:06:09.881664 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-fc7789ab-8276-4d1d-a6ca-5a6dca8f5cab tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:09.889072 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-fc7789ab-8276-4d1d-a6ca-5a6dca8f5cab tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: ffdd5acc-f9cf-4244-ae3b-429260b7c33f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:09.895904 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-fc7789ab-8276-4d1d-a6ca-5a6dca8f5cab tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:09.896351 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-fc7789ab-8276-4d1d-a6ca-5a6dca8f5cab tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:09.896784 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-fc7789ab-8276-4d1d-a6ca-5a6dca8f5cab tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:09.897523 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-fc7789ab-8276-4d1d-a6ca-5a6dca8f5cab tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:09.897967 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-fc7789ab-8276-4d1d-a6ca-5a6dca8f5cab tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:09.898402 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-fc7789ab-8276-4d1d-a6ca-5a6dca8f5cab tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:10.048004 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-fc7789ab-8276-4d1d-a6ca-5a6dca8f5cab tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:10.056389 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-fc7789ab-8276-4d1d-a6ca-5a6dca8f5cab tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/ffdd5acc-f9cf-4244-ae3b-429260b7c33f" status: 200 len: 1367 microversion: 2.1 time: 0.179114 Oct 30 01:06:10.071783 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 543/1087] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:06:09 2022] GET /compute/v2.1/servers/ffdd5acc-f9cf-4244-ae3b-429260b7c33f => generated 1367 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:06:10.406846 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-f01670f9-a1b7-403b-a77a-9661860718c7 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:10.407903 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-f01670f9-a1b7-403b-a77a-9661860718c7 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:10.415170 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f01670f9-a1b7-403b-a77a-9661860718c7 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:10.415645 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f01670f9-a1b7-403b-a77a-9661860718c7 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:10.416119 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f01670f9-a1b7-403b-a77a-9661860718c7 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:10.416864 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f01670f9-a1b7-403b-a77a-9661860718c7 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:10.417320 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f01670f9-a1b7-403b-a77a-9661860718c7 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:10.417750 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f01670f9-a1b7-403b-a77a-9661860718c7 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:10.553631 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-f01670f9-a1b7-403b-a77a-9661860718c7 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:10.565967 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-f01670f9-a1b7-403b-a77a-9661860718c7 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:10.568291 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-f01670f9-a1b7-403b-a77a-9661860718c7 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:10.569463 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-f01670f9-a1b7-403b-a77a-9661860718c7 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1744 microversion: 2.20 time: 0.166812 Oct 30 01:06:10.569991 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 545/1088] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:06:10 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1744 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:06:11.081879 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-dba5c211-ef9e-40f5-9f40-faef2ccf9fc3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:11.082473 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-dba5c211-ef9e-40f5-9f40-faef2ccf9fc3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: ffdd5acc-f9cf-4244-ae3b-429260b7c33f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:11.084512 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell0:SELECT=2 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:06:11.085179 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell0:SELECT=6 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:06:11.092619 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dba5c211-ef9e-40f5-9f40-faef2ccf9fc3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:11.093172 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dba5c211-ef9e-40f5-9f40-faef2ccf9fc3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:11.093694 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dba5c211-ef9e-40f5-9f40-faef2ccf9fc3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:11.094757 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dba5c211-ef9e-40f5-9f40-faef2ccf9fc3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:11.095459 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dba5c211-ef9e-40f5-9f40-faef2ccf9fc3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:11.096170 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dba5c211-ef9e-40f5-9f40-faef2ccf9fc3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:11.102283 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell1:SELECT=96,nova_cell1:INSERT=2,nova_cell1:UPDATE=2 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:06:11.235490 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-dba5c211-ef9e-40f5-9f40-faef2ccf9fc3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:11.243657 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-dba5c211-ef9e-40f5-9f40-faef2ccf9fc3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/ffdd5acc-f9cf-4244-ae3b-429260b7c33f" status: 200 len: 1367 microversion: 2.1 time: 0.165903 Oct 30 01:06:11.244258 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 544/1089] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:06:11 2022] GET /compute/v2.1/servers/ffdd5acc-f9cf-4244-ae3b-429260b7c33f => generated 1367 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:06:11.595679 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-4866a4ce-df89-4fba-9cc2-28eb3e2c2d68 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:11.596320 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-4866a4ce-df89-4fba-9cc2-28eb3e2c2d68 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:11.602928 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4866a4ce-df89-4fba-9cc2-28eb3e2c2d68 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:11.603401 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4866a4ce-df89-4fba-9cc2-28eb3e2c2d68 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:11.603953 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4866a4ce-df89-4fba-9cc2-28eb3e2c2d68 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:11.604648 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4866a4ce-df89-4fba-9cc2-28eb3e2c2d68 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:11.605101 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4866a4ce-df89-4fba-9cc2-28eb3e2c2d68 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:11.605520 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4866a4ce-df89-4fba-9cc2-28eb3e2c2d68 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:11.752158 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-4866a4ce-df89-4fba-9cc2-28eb3e2c2d68 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:11.756054 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell0:SELECT=6 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:06:11.762087 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-4866a4ce-df89-4fba-9cc2-28eb3e2c2d68 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:11.763602 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-4866a4ce-df89-4fba-9cc2-28eb3e2c2d68 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:11.764882 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-4866a4ce-df89-4fba-9cc2-28eb3e2c2d68 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1744 microversion: 2.20 time: 0.173193 Oct 30 01:06:11.765434 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 546/1090] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:06:11 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1744 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:06:12.268405 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-776faf09-2e27-4c86-980d-bc2b0caa30f3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:12.268976 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-776faf09-2e27-4c86-980d-bc2b0caa30f3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: ffdd5acc-f9cf-4244-ae3b-429260b7c33f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:12.276153 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-776faf09-2e27-4c86-980d-bc2b0caa30f3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:12.276706 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-776faf09-2e27-4c86-980d-bc2b0caa30f3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:12.277154 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-776faf09-2e27-4c86-980d-bc2b0caa30f3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:12.278064 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-776faf09-2e27-4c86-980d-bc2b0caa30f3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:12.278541 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-776faf09-2e27-4c86-980d-bc2b0caa30f3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:12.278975 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-776faf09-2e27-4c86-980d-bc2b0caa30f3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:12.410718 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-776faf09-2e27-4c86-980d-bc2b0caa30f3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:12.418483 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-776faf09-2e27-4c86-980d-bc2b0caa30f3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/ffdd5acc-f9cf-4244-ae3b-429260b7c33f" status: 200 len: 1367 microversion: 2.1 time: 0.154009 Oct 30 01:06:12.419177 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 545/1091] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:06:12 2022] GET /compute/v2.1/servers/ffdd5acc-f9cf-4244-ae3b-429260b7c33f => generated 1367 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:06:12.502248 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [req-52cd6a8d-7fbe-4d49-b82f-dfefe70d7362 req-ddb8f6fe-6bad-4148-969f-38387196a824 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ffdd5acc-f9cf-4244-ae3b-429260b7c33f", "name": "network-vif-plugged", "status": "completed", "tag": "c36e47a4-5169-485f-88ec-96caef1c3d0f"}]} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:06:12.513087 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-52cd6a8d-7fbe-4d49-b82f-dfefe70d7362 req-ddb8f6fe-6bad-4148-969f-38387196a824 service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:12.513730 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-52cd6a8d-7fbe-4d49-b82f-dfefe70d7362 req-ddb8f6fe-6bad-4148-969f-38387196a824 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:12.514268 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-52cd6a8d-7fbe-4d49-b82f-dfefe70d7362 req-ddb8f6fe-6bad-4148-969f-38387196a824 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:12.526526 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.compute.server_external_events [req-52cd6a8d-7fbe-4d49-b82f-dfefe70d7362 req-ddb8f6fe-6bad-4148-969f-38387196a824 service nova] Creating event network-vif-plugged:c36e47a4-5169-485f-88ec-96caef1c3d0f for instance ffdd5acc-f9cf-4244-ae3b-429260b7c33f on hv-cinder-maste Oct 30 01:06:12.531672 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [req-52cd6a8d-7fbe-4d49-b82f-dfefe70d7362 req-ddb8f6fe-6bad-4148-969f-38387196a824 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033270 Oct 30 01:06:12.532706 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 547/1092] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:06:12 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:06:12.791698 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-5419149f-7463-4161-9052-25b00153a6cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:12.792381 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-5419149f-7463-4161-9052-25b00153a6cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:12.798979 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5419149f-7463-4161-9052-25b00153a6cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:12.799465 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5419149f-7463-4161-9052-25b00153a6cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:12.800462 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5419149f-7463-4161-9052-25b00153a6cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:12.801666 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5419149f-7463-4161-9052-25b00153a6cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:12.802330 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5419149f-7463-4161-9052-25b00153a6cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:12.802957 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5419149f-7463-4161-9052-25b00153a6cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:12.950758 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-5419149f-7463-4161-9052-25b00153a6cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:12.961238 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-5419149f-7463-4161-9052-25b00153a6cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:12.963642 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-5419149f-7463-4161-9052-25b00153a6cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:12.965217 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-5419149f-7463-4161-9052-25b00153a6cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1744 microversion: 2.20 time: 0.177674 Oct 30 01:06:12.966406 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 546/1093] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:06:12 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1744 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:06:13.448320 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-97ccc70b-7dc3-4a8a-aaeb-6e7006d9eed3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:13.449000 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-97ccc70b-7dc3-4a8a-aaeb-6e7006d9eed3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: ffdd5acc-f9cf-4244-ae3b-429260b7c33f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:13.455307 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-97ccc70b-7dc3-4a8a-aaeb-6e7006d9eed3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:13.455805 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-97ccc70b-7dc3-4a8a-aaeb-6e7006d9eed3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:13.456234 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-97ccc70b-7dc3-4a8a-aaeb-6e7006d9eed3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:13.456923 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-97ccc70b-7dc3-4a8a-aaeb-6e7006d9eed3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:13.457364 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-97ccc70b-7dc3-4a8a-aaeb-6e7006d9eed3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:13.457781 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-97ccc70b-7dc3-4a8a-aaeb-6e7006d9eed3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:13.586348 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-97ccc70b-7dc3-4a8a-aaeb-6e7006d9eed3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:13.595071 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-97ccc70b-7dc3-4a8a-aaeb-6e7006d9eed3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/ffdd5acc-f9cf-4244-ae3b-429260b7c33f" status: 200 len: 1367 microversion: 2.1 time: 0.154926 Oct 30 01:06:13.595608 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 548/1094] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:06:13 2022] GET /compute/v2.1/servers/ffdd5acc-f9cf-4244-ae3b-429260b7c33f => generated 1367 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:06:13.996458 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-50ad2571-a660-4818-98b0-d32a79eccd78 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:13.997443 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-50ad2571-a660-4818-98b0-d32a79eccd78 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:14.008226 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-50ad2571-a660-4818-98b0-d32a79eccd78 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:14.009125 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-50ad2571-a660-4818-98b0-d32a79eccd78 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:14.009710 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-50ad2571-a660-4818-98b0-d32a79eccd78 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:14.010396 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-50ad2571-a660-4818-98b0-d32a79eccd78 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:14.010841 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-50ad2571-a660-4818-98b0-d32a79eccd78 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:14.011234 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-50ad2571-a660-4818-98b0-d32a79eccd78 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:14.147427 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-50ad2571-a660-4818-98b0-d32a79eccd78 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:14.156216 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-50ad2571-a660-4818-98b0-d32a79eccd78 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:14.157621 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-50ad2571-a660-4818-98b0-d32a79eccd78 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:14.158818 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-50ad2571-a660-4818-98b0-d32a79eccd78 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1744 microversion: 2.20 time: 0.169066 Oct 30 01:06:14.159419 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 547/1095] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:06:13 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1744 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:06:14.546265 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [req-88265687-1fb5-486c-bc7b-52ba014c6443 req-9a8f4ac0-6d80-47ea-bf02-a86f2b96daca service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f4033e06-e2d3-4abb-ba25-eddfaf3e7e20", "name": "network-vif-plugged", "status": "completed", "tag": "e5d64f79-b32b-48ac-8231-fbd53262a835"}]} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:06:14.557256 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-88265687-1fb5-486c-bc7b-52ba014c6443 req-9a8f4ac0-6d80-47ea-bf02-a86f2b96daca service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:14.557718 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-88265687-1fb5-486c-bc7b-52ba014c6443 req-9a8f4ac0-6d80-47ea-bf02-a86f2b96daca service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:14.558210 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-88265687-1fb5-486c-bc7b-52ba014c6443 req-9a8f4ac0-6d80-47ea-bf02-a86f2b96daca service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:14.569789 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.compute.server_external_events [req-88265687-1fb5-486c-bc7b-52ba014c6443 req-9a8f4ac0-6d80-47ea-bf02-a86f2b96daca service nova] Creating event network-vif-plugged:e5d64f79-b32b-48ac-8231-fbd53262a835 for instance f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 on hv-cinder-maste Oct 30 01:06:14.575948 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [req-88265687-1fb5-486c-bc7b-52ba014c6443 req-9a8f4ac0-6d80-47ea-bf02-a86f2b96daca service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033386 Oct 30 01:06:14.579880 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 549/1096] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:06:14 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:06:14.626274 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-488f2207-168c-4de9-8a57-c6926b399bd2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:14.626899 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-488f2207-168c-4de9-8a57-c6926b399bd2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: ffdd5acc-f9cf-4244-ae3b-429260b7c33f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:14.632947 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-488f2207-168c-4de9-8a57-c6926b399bd2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:14.633444 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-488f2207-168c-4de9-8a57-c6926b399bd2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:14.633872 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-488f2207-168c-4de9-8a57-c6926b399bd2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:14.634644 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-488f2207-168c-4de9-8a57-c6926b399bd2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:14.635130 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-488f2207-168c-4de9-8a57-c6926b399bd2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:14.635548 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-488f2207-168c-4de9-8a57-c6926b399bd2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:14.806381 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-488f2207-168c-4de9-8a57-c6926b399bd2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:14.814794 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-488f2207-168c-4de9-8a57-c6926b399bd2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/ffdd5acc-f9cf-4244-ae3b-429260b7c33f" status: 200 len: 1367 microversion: 2.1 time: 0.192213 Oct 30 01:06:14.815350 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 548/1097] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:06:14 2022] GET /compute/v2.1/servers/ffdd5acc-f9cf-4244-ae3b-429260b7c33f => generated 1367 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:06:15.185443 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-84c6f36d-c1d2-4609-a199-b2d123d83722 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:15.186160 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-84c6f36d-c1d2-4609-a199-b2d123d83722 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:15.193003 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-84c6f36d-c1d2-4609-a199-b2d123d83722 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:15.193494 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-84c6f36d-c1d2-4609-a199-b2d123d83722 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:15.193928 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-84c6f36d-c1d2-4609-a199-b2d123d83722 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:15.194638 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-84c6f36d-c1d2-4609-a199-b2d123d83722 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:15.195300 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-84c6f36d-c1d2-4609-a199-b2d123d83722 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:15.195502 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-84c6f36d-c1d2-4609-a199-b2d123d83722 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:15.348623 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-84c6f36d-c1d2-4609-a199-b2d123d83722 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:15.357852 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-84c6f36d-c1d2-4609-a199-b2d123d83722 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:15.359411 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-84c6f36d-c1d2-4609-a199-b2d123d83722 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:15.360500 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-84c6f36d-c1d2-4609-a199-b2d123d83722 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1744 microversion: 2.20 time: 0.179661 Oct 30 01:06:15.361055 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 550/1098] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:06:15 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1744 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 30 01:06:15.837943 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-ce006786-4905-42f2-907d-1feb69d54cf9 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:15.838527 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-ce006786-4905-42f2-907d-1feb69d54cf9 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: ffdd5acc-f9cf-4244-ae3b-429260b7c33f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:15.845580 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ce006786-4905-42f2-907d-1feb69d54cf9 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:15.846049 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ce006786-4905-42f2-907d-1feb69d54cf9 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:15.846467 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ce006786-4905-42f2-907d-1feb69d54cf9 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:15.847147 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ce006786-4905-42f2-907d-1feb69d54cf9 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:15.847754 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ce006786-4905-42f2-907d-1feb69d54cf9 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:15.848241 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ce006786-4905-42f2-907d-1feb69d54cf9 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:15.985914 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-ce006786-4905-42f2-907d-1feb69d54cf9 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:15.994072 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-ce006786-4905-42f2-907d-1feb69d54cf9 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/ffdd5acc-f9cf-4244-ae3b-429260b7c33f" status: 200 len: 1557 microversion: 2.1 time: 0.160023 Oct 30 01:06:15.997684 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 549/1099] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:06:15 2022] GET /compute/v2.1/servers/ffdd5acc-f9cf-4244-ae3b-429260b7c33f => generated 1557 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:06:16.016487 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-ea786526-2f65-4e5c-bc51-66216a5fde5f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:16.016974 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-ea786526-2f65-4e5c-bc51-66216a5fde5f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: ffdd5acc-f9cf-4244-ae3b-429260b7c33f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:16.023672 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ea786526-2f65-4e5c-bc51-66216a5fde5f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:16.024504 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ea786526-2f65-4e5c-bc51-66216a5fde5f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:16.025084 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ea786526-2f65-4e5c-bc51-66216a5fde5f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:16.025789 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ea786526-2f65-4e5c-bc51-66216a5fde5f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:16.026235 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ea786526-2f65-4e5c-bc51-66216a5fde5f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:16.027012 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ea786526-2f65-4e5c-bc51-66216a5fde5f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:16.108929 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-ea786526-2f65-4e5c-bc51-66216a5fde5f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/ffdd5acc-f9cf-4244-ae3b-429260b7c33f/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.096304 Oct 30 01:06:16.109529 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 551/1100] 10.222.0.16 () {58 vars in 1275 bytes} [Sun Oct 30 01:06:16 2022] GET /compute/v2.1/servers/ffdd5acc-f9cf-4244-ae3b-429260b7c33f/os-interface => generated 284 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:06:16.387436 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-bac5d397-0be8-4316-9535-a7c2e86cf268 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:16.388517 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-bac5d397-0be8-4316-9535-a7c2e86cf268 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:16.395239 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bac5d397-0be8-4316-9535-a7c2e86cf268 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:16.395959 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bac5d397-0be8-4316-9535-a7c2e86cf268 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:16.396552 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bac5d397-0be8-4316-9535-a7c2e86cf268 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:16.397603 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bac5d397-0be8-4316-9535-a7c2e86cf268 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:16.398196 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bac5d397-0be8-4316-9535-a7c2e86cf268 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:16.398761 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bac5d397-0be8-4316-9535-a7c2e86cf268 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:16.546582 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-bac5d397-0be8-4316-9535-a7c2e86cf268 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:16.555424 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-bac5d397-0be8-4316-9535-a7c2e86cf268 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:16.557126 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-bac5d397-0be8-4316-9535-a7c2e86cf268 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:16.558428 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-bac5d397-0be8-4316-9535-a7c2e86cf268 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1744 microversion: 2.20 time: 0.175000 Oct 30 01:06:16.559051 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 550/1101] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:06:16 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1744 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:06:17.094812 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [req-9c131efa-7f98-4d9b-b1a5-1b945be7c0ad req-3e3f25c6-6996-40bf-9656-6406b37accdd service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ffdd5acc-f9cf-4244-ae3b-429260b7c33f", "tag": "c36e47a4-5169-485f-88ec-96caef1c3d0f"}]} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:06:17.105282 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-9c131efa-7f98-4d9b-b1a5-1b945be7c0ad req-3e3f25c6-6996-40bf-9656-6406b37accdd service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:17.105841 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-9c131efa-7f98-4d9b-b1a5-1b945be7c0ad req-3e3f25c6-6996-40bf-9656-6406b37accdd service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:17.106455 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-9c131efa-7f98-4d9b-b1a5-1b945be7c0ad req-3e3f25c6-6996-40bf-9656-6406b37accdd service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:17.122886 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.compute.server_external_events [req-9c131efa-7f98-4d9b-b1a5-1b945be7c0ad req-3e3f25c6-6996-40bf-9656-6406b37accdd service nova] Creating event network-changed:c36e47a4-5169-485f-88ec-96caef1c3d0f for instance ffdd5acc-f9cf-4244-ae3b-429260b7c33f on hv-cinder-maste Oct 30 01:06:17.128616 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [req-9c131efa-7f98-4d9b-b1a5-1b945be7c0ad req-3e3f25c6-6996-40bf-9656-6406b37accdd service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038018 Oct 30 01:06:17.129503 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 552/1102] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:06:17 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:06:17.584392 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-d6e586f0-7bbd-4c4c-b1a9-bbc2bdba4325 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:17.585785 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-d6e586f0-7bbd-4c4c-b1a9-bbc2bdba4325 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:17.594308 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d6e586f0-7bbd-4c4c-b1a9-bbc2bdba4325 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:17.594956 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d6e586f0-7bbd-4c4c-b1a9-bbc2bdba4325 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:17.595744 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d6e586f0-7bbd-4c4c-b1a9-bbc2bdba4325 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:17.596631 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d6e586f0-7bbd-4c4c-b1a9-bbc2bdba4325 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:17.597237 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d6e586f0-7bbd-4c4c-b1a9-bbc2bdba4325 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:17.597817 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d6e586f0-7bbd-4c4c-b1a9-bbc2bdba4325 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:17.763791 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-d6e586f0-7bbd-4c4c-b1a9-bbc2bdba4325 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:17.774739 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-d6e586f0-7bbd-4c4c-b1a9-bbc2bdba4325 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:17.777413 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-d6e586f0-7bbd-4c4c-b1a9-bbc2bdba4325 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:17.780104 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-d6e586f0-7bbd-4c4c-b1a9-bbc2bdba4325 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1744 microversion: 2.20 time: 0.199437 Oct 30 01:06:17.781623 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 551/1103] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:06:17 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1744 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:06:18.809252 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-59ea44cb-19e4-43a4-adf2-d63eb91b10cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:18.815588 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-59ea44cb-19e4-43a4-adf2-d63eb91b10cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:18.828162 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-59ea44cb-19e4-43a4-adf2-d63eb91b10cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:18.828918 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-59ea44cb-19e4-43a4-adf2-d63eb91b10cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:18.829614 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-59ea44cb-19e4-43a4-adf2-d63eb91b10cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:18.830529 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-59ea44cb-19e4-43a4-adf2-d63eb91b10cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:18.831119 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-59ea44cb-19e4-43a4-adf2-d63eb91b10cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:18.836871 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-59ea44cb-19e4-43a4-adf2-d63eb91b10cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:18.987616 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-59ea44cb-19e4-43a4-adf2-d63eb91b10cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:18.997761 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-59ea44cb-19e4-43a4-adf2-d63eb91b10cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:18.999216 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-59ea44cb-19e4-43a4-adf2-d63eb91b10cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:19.000508 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-59ea44cb-19e4-43a4-adf2-d63eb91b10cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1739 microversion: 2.20 time: 0.198323 Oct 30 01:06:19.001241 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 553/1104] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:06:18 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1739 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:06:20.028407 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-f697f691-819c-4ae6-b05f-81714e132878 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:20.029925 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-f697f691-819c-4ae6-b05f-81714e132878 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:20.048552 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f697f691-819c-4ae6-b05f-81714e132878 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:20.049644 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f697f691-819c-4ae6-b05f-81714e132878 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:20.050328 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f697f691-819c-4ae6-b05f-81714e132878 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:20.051285 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f697f691-819c-4ae6-b05f-81714e132878 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:20.051826 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f697f691-819c-4ae6-b05f-81714e132878 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:20.052442 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f697f691-819c-4ae6-b05f-81714e132878 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:20.214638 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-f697f691-819c-4ae6-b05f-81714e132878 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:20.229624 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-f697f691-819c-4ae6-b05f-81714e132878 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:20.231881 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-f697f691-819c-4ae6-b05f-81714e132878 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:20.235935 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-f697f691-819c-4ae6-b05f-81714e132878 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1739 microversion: 2.20 time: 0.213419 Oct 30 01:06:20.236754 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 552/1105] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:06:20 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1739 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:06:21.262529 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-cd60b647-8f3b-4ebc-be3a-1e83ec2afe58 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:21.263186 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-cd60b647-8f3b-4ebc-be3a-1e83ec2afe58 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:21.270532 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cd60b647-8f3b-4ebc-be3a-1e83ec2afe58 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:21.271023 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cd60b647-8f3b-4ebc-be3a-1e83ec2afe58 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:21.271486 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cd60b647-8f3b-4ebc-be3a-1e83ec2afe58 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:21.272171 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cd60b647-8f3b-4ebc-be3a-1e83ec2afe58 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:21.272788 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cd60b647-8f3b-4ebc-be3a-1e83ec2afe58 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:21.273456 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cd60b647-8f3b-4ebc-be3a-1e83ec2afe58 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:21.443187 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-cd60b647-8f3b-4ebc-be3a-1e83ec2afe58 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:21.451609 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-cd60b647-8f3b-4ebc-be3a-1e83ec2afe58 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:21.453490 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-cd60b647-8f3b-4ebc-be3a-1e83ec2afe58 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:21.454875 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-cd60b647-8f3b-4ebc-be3a-1e83ec2afe58 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1739 microversion: 2.20 time: 0.196477 Oct 30 01:06:21.455438 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 554/1106] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:06:21 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1739 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:06:22.482214 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-0cb85b05-de9f-4fae-aea3-e139b8faf2bc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:22.483014 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-0cb85b05-de9f-4fae-aea3-e139b8faf2bc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:22.490624 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0cb85b05-de9f-4fae-aea3-e139b8faf2bc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:22.491107 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0cb85b05-de9f-4fae-aea3-e139b8faf2bc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:22.491537 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0cb85b05-de9f-4fae-aea3-e139b8faf2bc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:22.492261 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0cb85b05-de9f-4fae-aea3-e139b8faf2bc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:22.492917 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0cb85b05-de9f-4fae-aea3-e139b8faf2bc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:22.493330 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0cb85b05-de9f-4fae-aea3-e139b8faf2bc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:22.638325 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-0cb85b05-de9f-4fae-aea3-e139b8faf2bc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:22.647028 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-0cb85b05-de9f-4fae-aea3-e139b8faf2bc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:22.648758 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-0cb85b05-de9f-4fae-aea3-e139b8faf2bc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:22.649962 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-0cb85b05-de9f-4fae-aea3-e139b8faf2bc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1739 microversion: 2.20 time: 0.172001 Oct 30 01:06:22.650531 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 553/1107] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:06:22 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1739 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:06:23.678872 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-ce635092-87d2-4f40-83c7-3d13feb42020 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:23.680198 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-ce635092-87d2-4f40-83c7-3d13feb42020 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:23.690512 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ce635092-87d2-4f40-83c7-3d13feb42020 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:23.690911 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ce635092-87d2-4f40-83c7-3d13feb42020 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:23.691353 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ce635092-87d2-4f40-83c7-3d13feb42020 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:23.692113 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ce635092-87d2-4f40-83c7-3d13feb42020 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:23.692550 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ce635092-87d2-4f40-83c7-3d13feb42020 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:23.692963 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ce635092-87d2-4f40-83c7-3d13feb42020 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:23.836589 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-ce635092-87d2-4f40-83c7-3d13feb42020 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:23.845771 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-ce635092-87d2-4f40-83c7-3d13feb42020 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:23.847196 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-ce635092-87d2-4f40-83c7-3d13feb42020 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:23.848406 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-ce635092-87d2-4f40-83c7-3d13feb42020 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1739 microversion: 2.20 time: 0.174218 Oct 30 01:06:23.848983 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 555/1108] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:06:23 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1739 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:06:24.873346 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-7aed4f49-96af-4133-aad2-caef0b0d99d1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:24.873926 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-7aed4f49-96af-4133-aad2-caef0b0d99d1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:24.880526 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7aed4f49-96af-4133-aad2-caef0b0d99d1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:24.881006 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7aed4f49-96af-4133-aad2-caef0b0d99d1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:24.881436 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7aed4f49-96af-4133-aad2-caef0b0d99d1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:24.882129 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7aed4f49-96af-4133-aad2-caef0b0d99d1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:24.882580 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7aed4f49-96af-4133-aad2-caef0b0d99d1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:24.883001 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7aed4f49-96af-4133-aad2-caef0b0d99d1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:25.016590 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-7aed4f49-96af-4133-aad2-caef0b0d99d1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:25.025190 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-7aed4f49-96af-4133-aad2-caef0b0d99d1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:25.026652 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-7aed4f49-96af-4133-aad2-caef0b0d99d1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:25.027850 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-7aed4f49-96af-4133-aad2-caef0b0d99d1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1739 microversion: 2.20 time: 0.158537 Oct 30 01:06:25.028410 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 554/1109] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:06:24 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1739 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:06:26.054241 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-36229644-7846-4d4d-a428-e229c8591fe7 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:26.054858 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-36229644-7846-4d4d-a428-e229c8591fe7 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:26.062373 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-36229644-7846-4d4d-a428-e229c8591fe7 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:26.062870 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-36229644-7846-4d4d-a428-e229c8591fe7 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:26.063426 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-36229644-7846-4d4d-a428-e229c8591fe7 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:26.064247 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-36229644-7846-4d4d-a428-e229c8591fe7 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:26.067497 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-36229644-7846-4d4d-a428-e229c8591fe7 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:26.068310 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-36229644-7846-4d4d-a428-e229c8591fe7 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:26.207264 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-36229644-7846-4d4d-a428-e229c8591fe7 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:26.216814 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-36229644-7846-4d4d-a428-e229c8591fe7 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:26.218255 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-36229644-7846-4d4d-a428-e229c8591fe7 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:26.219436 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-36229644-7846-4d4d-a428-e229c8591fe7 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1739 microversion: 2.20 time: 0.169626 Oct 30 01:06:26.220109 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 556/1110] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:06:26 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1739 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:06:27.245940 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-b98223ca-8cd6-46a9-b780-8abc914236c2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:27.247418 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-b98223ca-8cd6-46a9-b780-8abc914236c2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:27.258621 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b98223ca-8cd6-46a9-b780-8abc914236c2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:27.259116 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b98223ca-8cd6-46a9-b780-8abc914236c2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:27.259551 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b98223ca-8cd6-46a9-b780-8abc914236c2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:27.260377 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b98223ca-8cd6-46a9-b780-8abc914236c2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:27.260854 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b98223ca-8cd6-46a9-b780-8abc914236c2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:27.261530 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b98223ca-8cd6-46a9-b780-8abc914236c2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:27.407636 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-b98223ca-8cd6-46a9-b780-8abc914236c2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:27.417431 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-b98223ca-8cd6-46a9-b780-8abc914236c2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:27.419117 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-b98223ca-8cd6-46a9-b780-8abc914236c2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:27.420681 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-b98223ca-8cd6-46a9-b780-8abc914236c2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1739 microversion: 2.20 time: 0.179037 Oct 30 01:06:27.421462 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 555/1111] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:06:27 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1739 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:06:28.446690 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-65f1f08f-2602-4580-a40d-329f5cdeb61a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:28.447388 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-65f1f08f-2602-4580-a40d-329f5cdeb61a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:28.454934 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-65f1f08f-2602-4580-a40d-329f5cdeb61a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:28.455386 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-65f1f08f-2602-4580-a40d-329f5cdeb61a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:28.455864 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-65f1f08f-2602-4580-a40d-329f5cdeb61a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:28.456621 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-65f1f08f-2602-4580-a40d-329f5cdeb61a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:28.457066 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-65f1f08f-2602-4580-a40d-329f5cdeb61a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:28.457564 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-65f1f08f-2602-4580-a40d-329f5cdeb61a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:28.634035 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-65f1f08f-2602-4580-a40d-329f5cdeb61a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:28.643651 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-65f1f08f-2602-4580-a40d-329f5cdeb61a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:28.645101 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-65f1f08f-2602-4580-a40d-329f5cdeb61a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:28.646518 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-65f1f08f-2602-4580-a40d-329f5cdeb61a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1739 microversion: 2.20 time: 0.204030 Oct 30 01:06:28.663788 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 557/1112] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:06:28 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1739 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:06:29.675588 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-7880c229-f947-4753-a2d4-01af5412ee14 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:29.677975 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-7880c229-f947-4753-a2d4-01af5412ee14 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:29.688118 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7880c229-f947-4753-a2d4-01af5412ee14 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:29.689754 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7880c229-f947-4753-a2d4-01af5412ee14 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:29.690658 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7880c229-f947-4753-a2d4-01af5412ee14 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:29.691937 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7880c229-f947-4753-a2d4-01af5412ee14 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:29.693003 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7880c229-f947-4753-a2d4-01af5412ee14 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:29.694190 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7880c229-f947-4753-a2d4-01af5412ee14 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:29.871702 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-7880c229-f947-4753-a2d4-01af5412ee14 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:29.880731 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-7880c229-f947-4753-a2d4-01af5412ee14 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:29.882205 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-7880c229-f947-4753-a2d4-01af5412ee14 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:29.883458 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-7880c229-f947-4753-a2d4-01af5412ee14 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1739 microversion: 2.20 time: 0.214103 Oct 30 01:06:29.891747 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 556/1113] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:06:29 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1739 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:06:30.909897 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-8fc701ce-6c5b-42c0-bde0-345464b0fcba tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:30.910643 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-8fc701ce-6c5b-42c0-bde0-345464b0fcba tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:30.917680 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8fc701ce-6c5b-42c0-bde0-345464b0fcba tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:30.918166 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8fc701ce-6c5b-42c0-bde0-345464b0fcba tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:30.918607 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8fc701ce-6c5b-42c0-bde0-345464b0fcba tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:30.919284 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8fc701ce-6c5b-42c0-bde0-345464b0fcba tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:30.920051 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8fc701ce-6c5b-42c0-bde0-345464b0fcba tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:30.920222 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8fc701ce-6c5b-42c0-bde0-345464b0fcba tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:31.076805 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-8fc701ce-6c5b-42c0-bde0-345464b0fcba tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:31.086016 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-8fc701ce-6c5b-42c0-bde0-345464b0fcba tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:31.087437 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-8fc701ce-6c5b-42c0-bde0-345464b0fcba tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:31.089015 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-8fc701ce-6c5b-42c0-bde0-345464b0fcba tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1739 microversion: 2.20 time: 0.183517 Oct 30 01:06:31.090533 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 558/1114] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:06:30 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1739 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:06:32.114894 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-bb77c7e6-9566-40cd-8d68-3a51d4859255 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:32.115511 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-bb77c7e6-9566-40cd-8d68-3a51d4859255 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:32.122260 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bb77c7e6-9566-40cd-8d68-3a51d4859255 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:32.122713 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bb77c7e6-9566-40cd-8d68-3a51d4859255 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:32.123193 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bb77c7e6-9566-40cd-8d68-3a51d4859255 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:32.123918 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bb77c7e6-9566-40cd-8d68-3a51d4859255 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:32.124545 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bb77c7e6-9566-40cd-8d68-3a51d4859255 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:32.125139 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bb77c7e6-9566-40cd-8d68-3a51d4859255 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:32.263998 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-bb77c7e6-9566-40cd-8d68-3a51d4859255 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:32.272977 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-bb77c7e6-9566-40cd-8d68-3a51d4859255 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:32.274497 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-bb77c7e6-9566-40cd-8d68-3a51d4859255 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:32.275755 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-bb77c7e6-9566-40cd-8d68-3a51d4859255 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1739 microversion: 2.20 time: 0.164996 Oct 30 01:06:32.287918 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 557/1115] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:06:32 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1739 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:06:33.302909 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-9102048d-60d6-4305-aec8-a11a09916a5e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:33.303516 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-9102048d-60d6-4305-aec8-a11a09916a5e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:33.309824 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9102048d-60d6-4305-aec8-a11a09916a5e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:33.310557 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9102048d-60d6-4305-aec8-a11a09916a5e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:33.310992 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9102048d-60d6-4305-aec8-a11a09916a5e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:33.311687 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9102048d-60d6-4305-aec8-a11a09916a5e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:33.312153 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9102048d-60d6-4305-aec8-a11a09916a5e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:33.312778 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9102048d-60d6-4305-aec8-a11a09916a5e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:33.465977 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-9102048d-60d6-4305-aec8-a11a09916a5e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:33.474087 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-9102048d-60d6-4305-aec8-a11a09916a5e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:33.475510 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-9102048d-60d6-4305-aec8-a11a09916a5e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:33.476706 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-9102048d-60d6-4305-aec8-a11a09916a5e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1739 microversion: 2.20 time: 0.178172 Oct 30 01:06:33.477267 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 559/1116] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:06:33 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1739 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:06:34.506112 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-15243d55-eff1-4a89-82ff-ef1eae92cba5 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:34.506966 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-15243d55-eff1-4a89-82ff-ef1eae92cba5 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:34.510691 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_api:SELECT=40 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:06:34.519044 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-15243d55-eff1-4a89-82ff-ef1eae92cba5 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:34.520002 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-15243d55-eff1-4a89-82ff-ef1eae92cba5 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:34.520348 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-15243d55-eff1-4a89-82ff-ef1eae92cba5 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:34.521054 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-15243d55-eff1-4a89-82ff-ef1eae92cba5 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:34.521461 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-15243d55-eff1-4a89-82ff-ef1eae92cba5 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:34.521873 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-15243d55-eff1-4a89-82ff-ef1eae92cba5 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:34.664043 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-15243d55-eff1-4a89-82ff-ef1eae92cba5 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:34.672378 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-15243d55-eff1-4a89-82ff-ef1eae92cba5 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:34.673790 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-15243d55-eff1-4a89-82ff-ef1eae92cba5 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:34.674953 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-15243d55-eff1-4a89-82ff-ef1eae92cba5 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1739 microversion: 2.20 time: 0.175129 Oct 30 01:06:34.675537 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 558/1117] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:06:34 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1739 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:06:35.704445 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-39fbdc93-6284-4cef-8edf-b761e58ffceb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:35.705309 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-39fbdc93-6284-4cef-8edf-b761e58ffceb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:35.709221 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_api:SELECT=42 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:06:35.718402 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-39fbdc93-6284-4cef-8edf-b761e58ffceb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:35.719188 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-39fbdc93-6284-4cef-8edf-b761e58ffceb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:35.720089 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-39fbdc93-6284-4cef-8edf-b761e58ffceb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:35.721275 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-39fbdc93-6284-4cef-8edf-b761e58ffceb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:35.722018 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-39fbdc93-6284-4cef-8edf-b761e58ffceb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:35.722713 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-39fbdc93-6284-4cef-8edf-b761e58ffceb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:35.862286 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-39fbdc93-6284-4cef-8edf-b761e58ffceb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:35.870737 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-39fbdc93-6284-4cef-8edf-b761e58ffceb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:35.872186 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-39fbdc93-6284-4cef-8edf-b761e58ffceb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:35.873349 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-39fbdc93-6284-4cef-8edf-b761e58ffceb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1721 microversion: 2.20 time: 0.175694 Oct 30 01:06:35.873933 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 560/1118] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:06:35 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1721 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:06:36.904406 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-dadcc463-d68e-4813-a9c8-95a2b683c32b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:36.905364 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-dadcc463-d68e-4813-a9c8-95a2b683c32b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:36.914800 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dadcc463-d68e-4813-a9c8-95a2b683c32b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:36.915280 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dadcc463-d68e-4813-a9c8-95a2b683c32b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:36.915704 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dadcc463-d68e-4813-a9c8-95a2b683c32b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:36.916418 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dadcc463-d68e-4813-a9c8-95a2b683c32b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:36.916859 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dadcc463-d68e-4813-a9c8-95a2b683c32b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:36.917273 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dadcc463-d68e-4813-a9c8-95a2b683c32b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:37.065361 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-dadcc463-d68e-4813-a9c8-95a2b683c32b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:37.077113 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-dadcc463-d68e-4813-a9c8-95a2b683c32b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:37.079336 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-dadcc463-d68e-4813-a9c8-95a2b683c32b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:37.081065 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-dadcc463-d68e-4813-a9c8-95a2b683c32b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1721 microversion: 2.20 time: 0.183340 Oct 30 01:06:37.081797 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 559/1119] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:06:36 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1721 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:06:38.113511 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-a2211c99-6e34-4378-80b3-07a5195d6c2a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:38.114149 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-a2211c99-6e34-4378-80b3-07a5195d6c2a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:38.121547 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a2211c99-6e34-4378-80b3-07a5195d6c2a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:38.122030 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a2211c99-6e34-4378-80b3-07a5195d6c2a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:38.122554 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a2211c99-6e34-4378-80b3-07a5195d6c2a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:38.123258 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a2211c99-6e34-4378-80b3-07a5195d6c2a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:38.123711 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a2211c99-6e34-4378-80b3-07a5195d6c2a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:38.124171 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a2211c99-6e34-4378-80b3-07a5195d6c2a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:38.261846 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-a2211c99-6e34-4378-80b3-07a5195d6c2a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:38.270451 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-a2211c99-6e34-4378-80b3-07a5195d6c2a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:38.271936 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-a2211c99-6e34-4378-80b3-07a5195d6c2a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:38.273178 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-a2211c99-6e34-4378-80b3-07a5195d6c2a tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1724 microversion: 2.20 time: 0.164026 Oct 30 01:06:38.273749 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 561/1120] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:06:38 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1724 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:06:38.298758 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-1aa3682d-65b0-41f9-8a7a-81af8a5ff964 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "208da5fb-ca5f-4082-90d3-193573ccf9c3"}} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:06:38.301271 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-1aa3682d-65b0-41f9-8a7a-81af8a5ff964 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:38.309380 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1aa3682d-65b0-41f9-8a7a-81af8a5ff964 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:38.309911 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1aa3682d-65b0-41f9-8a7a-81af8a5ff964 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:38.310309 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1aa3682d-65b0-41f9-8a7a-81af8a5ff964 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:38.311016 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1aa3682d-65b0-41f9-8a7a-81af8a5ff964 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:38.311461 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1aa3682d-65b0-41f9-8a7a-81af8a5ff964 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:38.312006 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1aa3682d-65b0-41f9-8a7a-81af8a5ff964 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:38.385086 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [req-aa0bfc56-862c-4bb3-9081-e65284d8cfb1 req-59531c4c-9e36-4d67-b149-e1c8f2bbde76 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f4033e06-e2d3-4abb-ba25-eddfaf3e7e20", "name": "network-vif-unplugged", "status": "completed", "tag": "e5d64f79-b32b-48ac-8231-fbd53262a835"}]} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:06:38.395225 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-aa0bfc56-862c-4bb3-9081-e65284d8cfb1 req-59531c4c-9e36-4d67-b149-e1c8f2bbde76 service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:38.395798 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-aa0bfc56-862c-4bb3-9081-e65284d8cfb1 req-59531c4c-9e36-4d67-b149-e1c8f2bbde76 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:38.396283 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-aa0bfc56-862c-4bb3-9081-e65284d8cfb1 req-59531c4c-9e36-4d67-b149-e1c8f2bbde76 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:38.408372 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.compute.server_external_events [req-aa0bfc56-862c-4bb3-9081-e65284d8cfb1 req-59531c4c-9e36-4d67-b149-e1c8f2bbde76 service nova] Creating event network-vif-unplugged:e5d64f79-b32b-48ac-8231-fbd53262a835 for instance f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 on hv-cinder-maste Oct 30 01:06:38.414153 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [req-aa0bfc56-862c-4bb3-9081-e65284d8cfb1 req-59531c4c-9e36-4d67-b149-e1c8f2bbde76 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.032796 Oct 30 01:06:38.415013 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 562/1121] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:06:38 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:06:38.420463 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell1:SELECT=100 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:06:38.644186 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-1aa3682d-65b0-41f9-8a7a-81af8a5ff964 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "POST /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.349410 Oct 30 01:06:38.644808 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 560/1122] 10.222.0.16 () {62 vars in 1364 bytes} [Sun Oct 30 01:06:38 2022] POST /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20/os-volume_attachments => generated 188 bytes in 351 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 30 01:06:38.784581 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-523526dc-d8a5-45af-a042-6448a6581b90 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:38.785194 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-523526dc-d8a5-45af-a042-6448a6581b90 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:38.792117 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-523526dc-d8a5-45af-a042-6448a6581b90 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:38.792596 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-523526dc-d8a5-45af-a042-6448a6581b90 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:38.793026 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-523526dc-d8a5-45af-a042-6448a6581b90 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:38.793716 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-523526dc-d8a5-45af-a042-6448a6581b90 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:38.794164 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-523526dc-d8a5-45af-a042-6448a6581b90 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:38.794590 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-523526dc-d8a5-45af-a042-6448a6581b90 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:38.879351 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell1:SELECT=100 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:06:39.089212 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-523526dc-d8a5-45af-a042-6448a6581b90 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "DELETE /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20/os-volume_attachments/208da5fb-ca5f-4082-90d3-193573ccf9c3" status: 202 len: 0 microversion: 2.20 time: 0.309022 Oct 30 01:06:39.089824 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 563/1123] 10.222.0.16 () {60 vars in 1457 bytes} [Sun Oct 30 01:06:38 2022] DELETE /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20/os-volume_attachments/208da5fb-ca5f-4082-90d3-193573ccf9c3 => generated 0 bytes in 311 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 30 01:06:39.150869 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-d5d4d7da-b043-4156-a413-627281bcc2af tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:06:39.151475 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-d5d4d7da-b043-4156-a413-627281bcc2af tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:39.157542 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d5d4d7da-b043-4156-a413-627281bcc2af tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:39.158008 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d5d4d7da-b043-4156-a413-627281bcc2af tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:39.158431 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d5d4d7da-b043-4156-a413-627281bcc2af tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:39.159106 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d5d4d7da-b043-4156-a413-627281bcc2af tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:39.159555 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d5d4d7da-b043-4156-a413-627281bcc2af tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:39.159993 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d5d4d7da-b043-4156-a413-627281bcc2af tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:39.203512 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-d5d4d7da-b043-4156-a413-627281bcc2af tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=118962) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Oct 30 01:06:39.294527 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-d5d4d7da-b043-4156-a413-627281bcc2af tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "POST /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20/action" status: 202 len: 0 microversion: 2.20 time: 0.147641 Oct 30 01:06:39.295127 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 561/1124] 10.222.0.16 () {62 vars in 1319 bytes} [Sun Oct 30 01:06:39 2022] POST /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20/action => generated 0 bytes in 149 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) Oct 30 01:06:39.303515 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-b5687491-14b0-4ecc-80b9-6bfaae3ec06e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:39.304442 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-b5687491-14b0-4ecc-80b9-6bfaae3ec06e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:39.314050 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b5687491-14b0-4ecc-80b9-6bfaae3ec06e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:39.314542 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b5687491-14b0-4ecc-80b9-6bfaae3ec06e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:39.315058 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b5687491-14b0-4ecc-80b9-6bfaae3ec06e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:39.316192 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b5687491-14b0-4ecc-80b9-6bfaae3ec06e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:39.316636 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b5687491-14b0-4ecc-80b9-6bfaae3ec06e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:39.317092 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b5687491-14b0-4ecc-80b9-6bfaae3ec06e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:39.454700 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-b5687491-14b0-4ecc-80b9-6bfaae3ec06e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:39.464040 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-b5687491-14b0-4ecc-80b9-6bfaae3ec06e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:39.466222 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-b5687491-14b0-4ecc-80b9-6bfaae3ec06e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:39.467453 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-b5687491-14b0-4ecc-80b9-6bfaae3ec06e tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1672 microversion: 2.20 time: 0.167954 Oct 30 01:06:39.468320 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 564/1125] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:06:39 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1672 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:06:40.242968 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-5ff5c1c9-3c96-4c51-8ae2-ce88a0dfb2b9 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Action: 'action', calling method: >, body: {"removeSecurityGroup": {"name": "tempest-secgroup-smoke-1066949310"}} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:06:40.243532 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-5ff5c1c9-3c96-4c51-8ae2-ce88a0dfb2b9 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:40.249568 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5ff5c1c9-3c96-4c51-8ae2-ce88a0dfb2b9 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:40.250057 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5ff5c1c9-3c96-4c51-8ae2-ce88a0dfb2b9 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:40.250490 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5ff5c1c9-3c96-4c51-8ae2-ce88a0dfb2b9 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:40.251320 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5ff5c1c9-3c96-4c51-8ae2-ce88a0dfb2b9 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:40.251798 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5ff5c1c9-3c96-4c51-8ae2-ce88a0dfb2b9 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:40.252221 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5ff5c1c9-3c96-4c51-8ae2-ce88a0dfb2b9 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:40.381397 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.network.security_group_api [None req-5ff5c1c9-3c96-4c51-8ae2-ce88a0dfb2b9 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Removing security group 2f032d87-6dca-4e2a-afc8-c3c7a20bc0ff from port ad10e942-b527-4fbd-867f-e161a78a11bc Oct 30 01:06:40.492742 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-a178735a-5bf8-44a2-bc68-1fd93bc3cfa3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:40.494272 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-a178735a-5bf8-44a2-bc68-1fd93bc3cfa3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:40.501318 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a178735a-5bf8-44a2-bc68-1fd93bc3cfa3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:40.502070 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a178735a-5bf8-44a2-bc68-1fd93bc3cfa3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:40.502566 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a178735a-5bf8-44a2-bc68-1fd93bc3cfa3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:40.503347 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a178735a-5bf8-44a2-bc68-1fd93bc3cfa3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:40.503922 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a178735a-5bf8-44a2-bc68-1fd93bc3cfa3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:40.504409 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a178735a-5bf8-44a2-bc68-1fd93bc3cfa3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:40.720714 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-a178735a-5bf8-44a2-bc68-1fd93bc3cfa3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:40.729424 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-a178735a-5bf8-44a2-bc68-1fd93bc3cfa3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:40.730888 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-a178735a-5bf8-44a2-bc68-1fd93bc3cfa3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:40.732094 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-a178735a-5bf8-44a2-bc68-1fd93bc3cfa3 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1730 microversion: 2.20 time: 0.243626 Oct 30 01:06:40.732669 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 565/1126] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:06:40 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1730 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 30 01:06:40.780849 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-5ff5c1c9-3c96-4c51-8ae2-ce88a0dfb2b9 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "POST /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/action" status: 202 len: 0 microversion: 2.1 time: 0.543017 Oct 30 01:06:40.781470 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 562/1127] 10.222.0.16 () {60 vars in 1278 bytes} [Sun Oct 30 01:06:40 2022] POST /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/action => generated 0 bytes in 544 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 30 01:06:40.792281 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [req-7e6f5000-f3f6-4a0b-9151-278bbe83fdac req-961d0a13-9614-4b04-99d5-2bc95cdb43b2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6f5485fd-b825-45e7-a5be-eb96208c09b7", "tag": "ad10e942-b527-4fbd-867f-e161a78a11bc"}]} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:06:40.802598 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-7e6f5000-f3f6-4a0b-9151-278bbe83fdac req-961d0a13-9614-4b04-99d5-2bc95cdb43b2 service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:40.803158 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-7e6f5000-f3f6-4a0b-9151-278bbe83fdac req-961d0a13-9614-4b04-99d5-2bc95cdb43b2 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:40.803668 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-7e6f5000-f3f6-4a0b-9151-278bbe83fdac req-961d0a13-9614-4b04-99d5-2bc95cdb43b2 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:40.838350 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.compute.server_external_events [req-7e6f5000-f3f6-4a0b-9151-278bbe83fdac req-961d0a13-9614-4b04-99d5-2bc95cdb43b2 service nova] Creating event network-changed:ad10e942-b527-4fbd-867f-e161a78a11bc for instance 6f5485fd-b825-45e7-a5be-eb96208c09b7 on hv-cinder-maste Oct 30 01:06:40.844238 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [req-7e6f5000-f3f6-4a0b-9151-278bbe83fdac req-961d0a13-9614-4b04-99d5-2bc95cdb43b2 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.055922 Oct 30 01:06:40.845331 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 566/1128] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:06:40 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:06:41.759952 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-0b89a654-2ee2-4327-bb04-4db6873be58f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:41.760570 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-0b89a654-2ee2-4327-bb04-4db6873be58f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:41.767347 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0b89a654-2ee2-4327-bb04-4db6873be58f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:41.768152 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0b89a654-2ee2-4327-bb04-4db6873be58f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:41.768615 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0b89a654-2ee2-4327-bb04-4db6873be58f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:41.769366 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0b89a654-2ee2-4327-bb04-4db6873be58f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:41.769854 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0b89a654-2ee2-4327-bb04-4db6873be58f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:41.770293 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0b89a654-2ee2-4327-bb04-4db6873be58f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:42.217766 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-0b89a654-2ee2-4327-bb04-4db6873be58f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:42.227356 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-0b89a654-2ee2-4327-bb04-4db6873be58f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:42.228933 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-0b89a654-2ee2-4327-bb04-4db6873be58f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:42.230309 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-0b89a654-2ee2-4327-bb04-4db6873be58f tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1730 microversion: 2.20 time: 0.475435 Oct 30 01:06:42.231139 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 563/1129] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:06:41 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1730 bytes in 477 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:06:42.582582 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-5f6482a9-c901-41bc-bcec-28d5469daeb8 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:42.583416 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-5f6482a9-c901-41bc-bcec-28d5469daeb8 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:42.592630 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5f6482a9-c901-41bc-bcec-28d5469daeb8 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:42.593321 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5f6482a9-c901-41bc-bcec-28d5469daeb8 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:42.593820 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5f6482a9-c901-41bc-bcec-28d5469daeb8 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:42.594704 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5f6482a9-c901-41bc-bcec-28d5469daeb8 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:42.595485 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5f6482a9-c901-41bc-bcec-28d5469daeb8 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:42.596007 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5f6482a9-c901-41bc-bcec-28d5469daeb8 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:42.686744 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.objects.instance [None req-5f6482a9-c901-41bc-bcec-28d5469daeb8 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lazy-loading 'services' on Instance uuid 6f5485fd-b825-45e7-a5be-eb96208c09b7 {{(pid=118961) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 30 01:06:42.787157 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-5f6482a9-c901-41bc-bcec-28d5469daeb8 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "DELETE /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments/a125d045-591a-4811-a456-a51a7989295e" status: 202 len: 0 microversion: 2.1 time: 0.208447 Oct 30 01:06:42.789283 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 567/1130] 10.222.0.16 () {58 vars in 1416 bytes} [Sun Oct 30 01:06:42 2022] DELETE /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments/a125d045-591a-4811-a456-a51a7989295e => generated 0 bytes in 212 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 30 01:06:42.797562 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-1c48fda3-39a2-4b0a-934a-82bdc202c90a tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:42.799891 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-1c48fda3-39a2-4b0a-934a-82bdc202c90a tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:42.812274 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1c48fda3-39a2-4b0a-934a-82bdc202c90a tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:42.813045 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1c48fda3-39a2-4b0a-934a-82bdc202c90a tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:42.819771 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1c48fda3-39a2-4b0a-934a-82bdc202c90a tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:42.820787 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1c48fda3-39a2-4b0a-934a-82bdc202c90a tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:42.821410 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1c48fda3-39a2-4b0a-934a-82bdc202c90a tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:42.821991 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1c48fda3-39a2-4b0a-934a-82bdc202c90a tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:42.861876 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [req-fe0bddd6-d51e-4aff-b5c6-b9bc1e7a3a62 req-1c54d06e-c182-41ca-ac37-8b1b3b358b4e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6f5485fd-b825-45e7-a5be-eb96208c09b7", "tag": "ad10e942-b527-4fbd-867f-e161a78a11bc"}]} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:06:42.872518 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-1c48fda3-39a2-4b0a-934a-82bdc202c90a tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.078786 Oct 30 01:06:42.877954 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 564/1131] 10.222.0.16 () {58 vars in 1302 bytes} [Sun Oct 30 01:06:42 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments => generated 197 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:06:42.881499 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-fe0bddd6-d51e-4aff-b5c6-b9bc1e7a3a62 req-1c54d06e-c182-41ca-ac37-8b1b3b358b4e service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:42.882315 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-fe0bddd6-d51e-4aff-b5c6-b9bc1e7a3a62 req-1c54d06e-c182-41ca-ac37-8b1b3b358b4e service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:42.882743 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-fe0bddd6-d51e-4aff-b5c6-b9bc1e7a3a62 req-1c54d06e-c182-41ca-ac37-8b1b3b358b4e service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:42.912305 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.compute.server_external_events [req-fe0bddd6-d51e-4aff-b5c6-b9bc1e7a3a62 req-1c54d06e-c182-41ca-ac37-8b1b3b358b4e service nova] Creating event network-changed:ad10e942-b527-4fbd-867f-e161a78a11bc for instance 6f5485fd-b825-45e7-a5be-eb96208c09b7 on hv-cinder-maste Oct 30 01:06:42.927229 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [req-fe0bddd6-d51e-4aff-b5c6-b9bc1e7a3a62 req-1c54d06e-c182-41ca-ac37-8b1b3b358b4e service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.071277 Oct 30 01:06:42.928274 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 568/1132] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:06:42 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 30 01:06:43.256537 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-78d45185-1a0e-4501-a547-ef10f3ef6e53 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:43.257631 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-78d45185-1a0e-4501-a547-ef10f3ef6e53 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:43.264098 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-78d45185-1a0e-4501-a547-ef10f3ef6e53 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:43.264805 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-78d45185-1a0e-4501-a547-ef10f3ef6e53 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:43.265229 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-78d45185-1a0e-4501-a547-ef10f3ef6e53 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:43.266079 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-78d45185-1a0e-4501-a547-ef10f3ef6e53 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:43.266555 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-78d45185-1a0e-4501-a547-ef10f3ef6e53 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:43.267124 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-78d45185-1a0e-4501-a547-ef10f3ef6e53 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:43.426961 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-78d45185-1a0e-4501-a547-ef10f3ef6e53 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:43.439299 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-78d45185-1a0e-4501-a547-ef10f3ef6e53 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:43.440938 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-78d45185-1a0e-4501-a547-ef10f3ef6e53 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:43.442286 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-78d45185-1a0e-4501-a547-ef10f3ef6e53 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1730 microversion: 2.20 time: 0.189852 Oct 30 01:06:43.443368 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 565/1133] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:06:43 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1730 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:06:43.886910 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-51874ccd-0dcd-4f4c-977c-9f240b8632b2 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:43.889206 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-51874ccd-0dcd-4f4c-977c-9f240b8632b2 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:43.896025 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-51874ccd-0dcd-4f4c-977c-9f240b8632b2 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:43.896500 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-51874ccd-0dcd-4f4c-977c-9f240b8632b2 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:43.896930 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-51874ccd-0dcd-4f4c-977c-9f240b8632b2 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:43.897715 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-51874ccd-0dcd-4f4c-977c-9f240b8632b2 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:43.898160 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-51874ccd-0dcd-4f4c-977c-9f240b8632b2 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:43.898577 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-51874ccd-0dcd-4f4c-977c-9f240b8632b2 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:43.927645 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-51874ccd-0dcd-4f4c-977c-9f240b8632b2 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045100 Oct 30 01:06:43.928221 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 569/1134] 10.222.0.16 () {58 vars in 1302 bytes} [Sun Oct 30 01:06:43 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:06:44.470334 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-9fc7b81a-2268-4a72-8f7a-53827f16df97 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:44.471086 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-9fc7b81a-2268-4a72-8f7a-53827f16df97 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:44.478393 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9fc7b81a-2268-4a72-8f7a-53827f16df97 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:44.478876 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9fc7b81a-2268-4a72-8f7a-53827f16df97 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:44.479332 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9fc7b81a-2268-4a72-8f7a-53827f16df97 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:44.480089 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9fc7b81a-2268-4a72-8f7a-53827f16df97 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:44.480537 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9fc7b81a-2268-4a72-8f7a-53827f16df97 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:44.480960 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9fc7b81a-2268-4a72-8f7a-53827f16df97 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:44.625357 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-9fc7b81a-2268-4a72-8f7a-53827f16df97 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:44.634700 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-9fc7b81a-2268-4a72-8f7a-53827f16df97 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:44.636253 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-9fc7b81a-2268-4a72-8f7a-53827f16df97 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:44.637571 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-9fc7b81a-2268-4a72-8f7a-53827f16df97 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1730 microversion: 2.20 time: 0.172268 Oct 30 01:06:44.639807 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 566/1135] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:06:44 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1730 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:06:44.941697 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-aa836c1a-1116-4b46-9732-fb65b33f9ba7 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:44.944725 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-aa836c1a-1116-4b46-9732-fb65b33f9ba7 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:44.951696 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-aa836c1a-1116-4b46-9732-fb65b33f9ba7 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:44.952222 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-aa836c1a-1116-4b46-9732-fb65b33f9ba7 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:44.952658 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-aa836c1a-1116-4b46-9732-fb65b33f9ba7 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:44.953365 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-aa836c1a-1116-4b46-9732-fb65b33f9ba7 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:44.953814 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-aa836c1a-1116-4b46-9732-fb65b33f9ba7 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:44.954237 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-aa836c1a-1116-4b46-9732-fb65b33f9ba7 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:44.986079 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-aa836c1a-1116-4b46-9732-fb65b33f9ba7 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048496 Oct 30 01:06:44.986666 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 570/1136] 10.222.0.16 () {58 vars in 1302 bytes} [Sun Oct 30 01:06:44 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:06:45.663187 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-3f14966e-edec-4e99-9196-9cb20ea68d6d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:45.663944 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-3f14966e-edec-4e99-9196-9cb20ea68d6d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:45.670613 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3f14966e-edec-4e99-9196-9cb20ea68d6d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:45.671123 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3f14966e-edec-4e99-9196-9cb20ea68d6d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:45.671535 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3f14966e-edec-4e99-9196-9cb20ea68d6d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:45.672338 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3f14966e-edec-4e99-9196-9cb20ea68d6d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:45.672781 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3f14966e-edec-4e99-9196-9cb20ea68d6d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:45.673205 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3f14966e-edec-4e99-9196-9cb20ea68d6d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:45.821949 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-3f14966e-edec-4e99-9196-9cb20ea68d6d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:45.833813 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-3f14966e-edec-4e99-9196-9cb20ea68d6d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:45.836825 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-3f14966e-edec-4e99-9196-9cb20ea68d6d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:45.838818 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-3f14966e-edec-4e99-9196-9cb20ea68d6d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1730 microversion: 2.20 time: 0.179645 Oct 30 01:06:45.839409 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 567/1137] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:06:45 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1730 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:06:45.999272 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-2715fab7-e118-4c2f-9cdc-1209baa29737 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:46.001586 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-2715fab7-e118-4c2f-9cdc-1209baa29737 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:46.009151 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2715fab7-e118-4c2f-9cdc-1209baa29737 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:46.009629 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2715fab7-e118-4c2f-9cdc-1209baa29737 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:46.010064 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2715fab7-e118-4c2f-9cdc-1209baa29737 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:46.010794 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2715fab7-e118-4c2f-9cdc-1209baa29737 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:46.011243 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2715fab7-e118-4c2f-9cdc-1209baa29737 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:46.011991 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2715fab7-e118-4c2f-9cdc-1209baa29737 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:46.065680 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-2715fab7-e118-4c2f-9cdc-1209baa29737 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.070347 Oct 30 01:06:46.079079 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 571/1138] 10.222.0.16 () {58 vars in 1302 bytes} [Sun Oct 30 01:06:45 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments => generated 197 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:06:46.865516 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-c3590344-d636-4794-9c07-ddf244392302 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:46.866182 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-c3590344-d636-4794-9c07-ddf244392302 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:46.873422 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c3590344-d636-4794-9c07-ddf244392302 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:46.873877 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c3590344-d636-4794-9c07-ddf244392302 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:46.874302 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c3590344-d636-4794-9c07-ddf244392302 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:46.875011 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c3590344-d636-4794-9c07-ddf244392302 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:46.875457 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c3590344-d636-4794-9c07-ddf244392302 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:46.875931 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c3590344-d636-4794-9c07-ddf244392302 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:47.014863 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-c3590344-d636-4794-9c07-ddf244392302 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:47.023269 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-c3590344-d636-4794-9c07-ddf244392302 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:47.024733 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-c3590344-d636-4794-9c07-ddf244392302 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:47.025890 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-c3590344-d636-4794-9c07-ddf244392302 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1730 microversion: 2.20 time: 0.164434 Oct 30 01:06:47.026450 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 568/1139] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:06:46 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1730 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:06:47.091445 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-7478c975-e1f6-4a89-ab22-d440390c5111 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:47.093592 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-7478c975-e1f6-4a89-ab22-d440390c5111 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:47.100377 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7478c975-e1f6-4a89-ab22-d440390c5111 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:47.100875 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7478c975-e1f6-4a89-ab22-d440390c5111 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:47.101388 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7478c975-e1f6-4a89-ab22-d440390c5111 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:47.102130 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7478c975-e1f6-4a89-ab22-d440390c5111 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:47.102581 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7478c975-e1f6-4a89-ab22-d440390c5111 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:47.103011 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7478c975-e1f6-4a89-ab22-d440390c5111 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:47.136797 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-7478c975-e1f6-4a89-ab22-d440390c5111 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049845 Oct 30 01:06:47.137373 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 572/1140] 10.222.0.16 () {58 vars in 1302 bytes} [Sun Oct 30 01:06:47 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:06:48.062215 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-3f79cfda-f2a8-4eca-8ed3-21603e5fdeb7 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:48.062886 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-3f79cfda-f2a8-4eca-8ed3-21603e5fdeb7 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:48.069905 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3f79cfda-f2a8-4eca-8ed3-21603e5fdeb7 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:48.070774 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3f79cfda-f2a8-4eca-8ed3-21603e5fdeb7 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:48.071671 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3f79cfda-f2a8-4eca-8ed3-21603e5fdeb7 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:48.072667 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3f79cfda-f2a8-4eca-8ed3-21603e5fdeb7 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:48.073327 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3f79cfda-f2a8-4eca-8ed3-21603e5fdeb7 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:48.073973 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3f79cfda-f2a8-4eca-8ed3-21603e5fdeb7 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:48.151418 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-fb6244dd-d9b8-492c-b473-93b24226045f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:48.153644 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-fb6244dd-d9b8-492c-b473-93b24226045f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:48.164089 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-fb6244dd-d9b8-492c-b473-93b24226045f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:48.164089 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-fb6244dd-d9b8-492c-b473-93b24226045f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:48.164089 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-fb6244dd-d9b8-492c-b473-93b24226045f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:48.164089 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-fb6244dd-d9b8-492c-b473-93b24226045f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:48.164436 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-fb6244dd-d9b8-492c-b473-93b24226045f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:48.164436 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-fb6244dd-d9b8-492c-b473-93b24226045f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:48.200138 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-fb6244dd-d9b8-492c-b473-93b24226045f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054472 Oct 30 01:06:48.205266 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 573/1141] 10.222.0.16 () {58 vars in 1302 bytes} [Sun Oct 30 01:06:48 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 30 01:06:48.237957 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-3f79cfda-f2a8-4eca-8ed3-21603e5fdeb7 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:48.247191 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-3f79cfda-f2a8-4eca-8ed3-21603e5fdeb7 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:48.249155 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-3f79cfda-f2a8-4eca-8ed3-21603e5fdeb7 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:48.250649 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-3f79cfda-f2a8-4eca-8ed3-21603e5fdeb7 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1730 microversion: 2.20 time: 0.199828 Oct 30 01:06:48.250799 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 569/1142] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:06:48 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1730 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:06:48.401257 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-8a5c78e3-0c8c-4145-8d79-6c0df3c9d083 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:48.401701 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-8a5c78e3-0c8c-4145-8d79-6c0df3c9d083 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: d95ac907-83b9-4fe2-b49a-defbe7240d3a] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:48.407916 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8a5c78e3-0c8c-4145-8d79-6c0df3c9d083 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:48.408408 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8a5c78e3-0c8c-4145-8d79-6c0df3c9d083 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:48.408848 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8a5c78e3-0c8c-4145-8d79-6c0df3c9d083 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:48.409601 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8a5c78e3-0c8c-4145-8d79-6c0df3c9d083 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:48.410056 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8a5c78e3-0c8c-4145-8d79-6c0df3c9d083 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:48.410482 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8a5c78e3-0c8c-4145-8d79-6c0df3c9d083 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:48.436543 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-8a5c78e3-0c8c-4145-8d79-6c0df3c9d083 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a/ips" status: 200 len: 146 microversion: 2.1 time: 0.186567 Oct 30 01:06:48.437110 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 574/1143] 10.222.0.16 () {58 vars in 1248 bytes} [Sun Oct 30 01:06:48 2022] GET /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a/ips => generated 146 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:06:49.212971 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-4852d8fc-becb-4f47-ab5e-02095de3a0c0 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:49.221163 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-4852d8fc-becb-4f47-ab5e-02095de3a0c0 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:49.242358 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4852d8fc-becb-4f47-ab5e-02095de3a0c0 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:49.243011 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4852d8fc-becb-4f47-ab5e-02095de3a0c0 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:49.243844 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4852d8fc-becb-4f47-ab5e-02095de3a0c0 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:49.245037 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4852d8fc-becb-4f47-ab5e-02095de3a0c0 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:49.245037 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4852d8fc-becb-4f47-ab5e-02095de3a0c0 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:49.245399 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4852d8fc-becb-4f47-ab5e-02095de3a0c0 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:49.276824 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-2cddc166-b558-48e2-a87d-6b1865d522cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:49.277674 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-2cddc166-b558-48e2-a87d-6b1865d522cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:49.284101 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-4852d8fc-becb-4f47-ab5e-02095de3a0c0 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.075183 Oct 30 01:06:49.285260 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 570/1144] 10.222.0.16 () {58 vars in 1302 bytes} [Sun Oct 30 01:06:49 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments => generated 197 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:06:49.288885 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2cddc166-b558-48e2-a87d-6b1865d522cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:49.289619 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2cddc166-b558-48e2-a87d-6b1865d522cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:49.290433 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2cddc166-b558-48e2-a87d-6b1865d522cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:49.291449 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2cddc166-b558-48e2-a87d-6b1865d522cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:49.292134 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2cddc166-b558-48e2-a87d-6b1865d522cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:49.292745 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2cddc166-b558-48e2-a87d-6b1865d522cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:49.451352 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-2cddc166-b558-48e2-a87d-6b1865d522cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:49.464687 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-2cddc166-b558-48e2-a87d-6b1865d522cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:49.466509 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-2cddc166-b558-48e2-a87d-6b1865d522cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:49.472033 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-2cddc166-b558-48e2-a87d-6b1865d522cd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1730 microversion: 2.20 time: 0.196984 Oct 30 01:06:49.472684 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 575/1145] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:06:49 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1730 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 30 01:06:50.297308 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-f42d7bc1-e319-4838-a5b0-1974ca2626fe tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:50.299535 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-f42d7bc1-e319-4838-a5b0-1974ca2626fe tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:50.306539 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f42d7bc1-e319-4838-a5b0-1974ca2626fe tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:50.307023 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f42d7bc1-e319-4838-a5b0-1974ca2626fe tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:50.307446 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f42d7bc1-e319-4838-a5b0-1974ca2626fe tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:50.308186 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f42d7bc1-e319-4838-a5b0-1974ca2626fe tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:50.308644 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f42d7bc1-e319-4838-a5b0-1974ca2626fe tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:50.309055 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-f42d7bc1-e319-4838-a5b0-1974ca2626fe tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:50.340062 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-f42d7bc1-e319-4838-a5b0-1974ca2626fe tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046653 Oct 30 01:06:50.344485 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 571/1146] 10.222.0.16 () {58 vars in 1302 bytes} [Sun Oct 30 01:06:50 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:06:50.498740 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-10239ae0-3b2d-49a5-966b-a88ed9d72e42 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:50.500219 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-10239ae0-3b2d-49a5-966b-a88ed9d72e42 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:50.507305 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-10239ae0-3b2d-49a5-966b-a88ed9d72e42 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:50.507812 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-10239ae0-3b2d-49a5-966b-a88ed9d72e42 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:50.508240 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-10239ae0-3b2d-49a5-966b-a88ed9d72e42 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:50.508940 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-10239ae0-3b2d-49a5-966b-a88ed9d72e42 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:50.509453 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-10239ae0-3b2d-49a5-966b-a88ed9d72e42 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:50.510006 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-10239ae0-3b2d-49a5-966b-a88ed9d72e42 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:50.653139 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-10239ae0-3b2d-49a5-966b-a88ed9d72e42 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:50.665251 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-10239ae0-3b2d-49a5-966b-a88ed9d72e42 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:50.667300 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-10239ae0-3b2d-49a5-966b-a88ed9d72e42 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:50.668938 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-10239ae0-3b2d-49a5-966b-a88ed9d72e42 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1730 microversion: 2.20 time: 0.174324 Oct 30 01:06:50.670369 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 576/1147] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:06:50 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1730 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:06:50.889330 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-cfc91474-235a-4d6a-a6df-f1217451f724 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "90c927cc-c3d0-46e0-b10f-c90b6a95232a"}} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:06:50.891773 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-cfc91474-235a-4d6a-a6df-f1217451f724 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: d95ac907-83b9-4fe2-b49a-defbe7240d3a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:50.918947 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-cfc91474-235a-4d6a-a6df-f1217451f724 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:50.919435 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-cfc91474-235a-4d6a-a6df-f1217451f724 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:50.919904 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-cfc91474-235a-4d6a-a6df-f1217451f724 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:50.920617 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-cfc91474-235a-4d6a-a6df-f1217451f724 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:50.921096 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-cfc91474-235a-4d6a-a6df-f1217451f724 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:50.921511 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-cfc91474-235a-4d6a-a6df-f1217451f724 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:51.354599 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-08142ce6-9bd6-41ac-a718-78da4a1acf77 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:51.356970 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-08142ce6-9bd6-41ac-a718-78da4a1acf77 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:51.363772 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-08142ce6-9bd6-41ac-a718-78da4a1acf77 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:51.364081 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-08142ce6-9bd6-41ac-a718-78da4a1acf77 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:51.367553 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-08142ce6-9bd6-41ac-a718-78da4a1acf77 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:51.367553 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-08142ce6-9bd6-41ac-a718-78da4a1acf77 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:51.367553 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-08142ce6-9bd6-41ac-a718-78da4a1acf77 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:51.367553 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-08142ce6-9bd6-41ac-a718-78da4a1acf77 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:51.397386 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-08142ce6-9bd6-41ac-a718-78da4a1acf77 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048072 Oct 30 01:06:51.397946 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 577/1148] 10.222.0.16 () {58 vars in 1302 bytes} [Sun Oct 30 01:06:51 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:06:51.698293 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-55447677-80df-4709-b4f2-022d513ce918 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:51.698938 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-55447677-80df-4709-b4f2-022d513ce918 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:51.705619 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-55447677-80df-4709-b4f2-022d513ce918 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:51.706104 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-55447677-80df-4709-b4f2-022d513ce918 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:51.706533 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-55447677-80df-4709-b4f2-022d513ce918 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:51.707231 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-55447677-80df-4709-b4f2-022d513ce918 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:51.707674 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-55447677-80df-4709-b4f2-022d513ce918 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:51.708128 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-55447677-80df-4709-b4f2-022d513ce918 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:51.859234 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-55447677-80df-4709-b4f2-022d513ce918 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:51.867371 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-55447677-80df-4709-b4f2-022d513ce918 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:51.882716 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-55447677-80df-4709-b4f2-022d513ce918 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:51.884002 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-55447677-80df-4709-b4f2-022d513ce918 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1730 microversion: 2.20 time: 0.189929 Oct 30 01:06:51.885049 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 578/1149] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:06:51 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1730 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 30 01:06:52.410768 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-bd40d22d-6256-49d1-b11d-708352c5f5d5 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:52.413052 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-bd40d22d-6256-49d1-b11d-708352c5f5d5 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:52.419677 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bd40d22d-6256-49d1-b11d-708352c5f5d5 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:52.420334 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bd40d22d-6256-49d1-b11d-708352c5f5d5 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:52.420787 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bd40d22d-6256-49d1-b11d-708352c5f5d5 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:52.421520 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bd40d22d-6256-49d1-b11d-708352c5f5d5 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:52.421965 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bd40d22d-6256-49d1-b11d-708352c5f5d5 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:52.422391 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bd40d22d-6256-49d1-b11d-708352c5f5d5 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:52.436627 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell1:SELECT=93,nova_cell1:INSERT=3,nova_cell1:UPDATE=4 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:06:52.453314 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-bd40d22d-6256-49d1-b11d-708352c5f5d5 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046673 Oct 30 01:06:52.453871 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 579/1150] 10.222.0.16 () {58 vars in 1302 bytes} [Sun Oct 30 01:06:52 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:06:52.913822 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-6b7a7fff-b071-4c3e-88e8-3664a8352f08 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:52.914942 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-6b7a7fff-b071-4c3e-88e8-3664a8352f08 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:52.922610 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6b7a7fff-b071-4c3e-88e8-3664a8352f08 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:52.923021 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6b7a7fff-b071-4c3e-88e8-3664a8352f08 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:52.923468 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6b7a7fff-b071-4c3e-88e8-3664a8352f08 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:52.924168 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6b7a7fff-b071-4c3e-88e8-3664a8352f08 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:52.924610 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6b7a7fff-b071-4c3e-88e8-3664a8352f08 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:52.925028 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6b7a7fff-b071-4c3e-88e8-3664a8352f08 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:53.070671 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-6b7a7fff-b071-4c3e-88e8-3664a8352f08 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:53.078929 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-6b7a7fff-b071-4c3e-88e8-3664a8352f08 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:53.080390 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-6b7a7fff-b071-4c3e-88e8-3664a8352f08 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:53.081557 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-6b7a7fff-b071-4c3e-88e8-3664a8352f08 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1730 microversion: 2.20 time: 0.174540 Oct 30 01:06:53.082106 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 580/1151] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:06:52 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1730 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:06:53.468369 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-72b4bd1e-1ecf-4485-baf3-cf8cd1819501 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:53.470573 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-72b4bd1e-1ecf-4485-baf3-cf8cd1819501 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:53.476907 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-72b4bd1e-1ecf-4485-baf3-cf8cd1819501 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:53.477395 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-72b4bd1e-1ecf-4485-baf3-cf8cd1819501 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:53.477996 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-72b4bd1e-1ecf-4485-baf3-cf8cd1819501 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:53.478855 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-72b4bd1e-1ecf-4485-baf3-cf8cd1819501 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:53.479312 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-72b4bd1e-1ecf-4485-baf3-cf8cd1819501 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:53.479761 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-72b4bd1e-1ecf-4485-baf3-cf8cd1819501 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:53.514144 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-72b4bd1e-1ecf-4485-baf3-cf8cd1819501 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051608 Oct 30 01:06:53.514716 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 581/1152] 10.222.0.16 () {58 vars in 1302 bytes} [Sun Oct 30 01:06:53 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:06:54.108074 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-3c910fdf-475f-446d-9d92-c1532f63e47d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:54.108706 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-3c910fdf-475f-446d-9d92-c1532f63e47d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:54.115767 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3c910fdf-475f-446d-9d92-c1532f63e47d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:54.116275 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3c910fdf-475f-446d-9d92-c1532f63e47d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:54.116712 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3c910fdf-475f-446d-9d92-c1532f63e47d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:54.117435 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3c910fdf-475f-446d-9d92-c1532f63e47d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:54.117885 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3c910fdf-475f-446d-9d92-c1532f63e47d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:54.118307 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3c910fdf-475f-446d-9d92-c1532f63e47d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:54.253627 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-3c910fdf-475f-446d-9d92-c1532f63e47d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:54.262213 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-3c910fdf-475f-446d-9d92-c1532f63e47d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:54.263638 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-3c910fdf-475f-446d-9d92-c1532f63e47d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:54.264838 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-3c910fdf-475f-446d-9d92-c1532f63e47d tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1730 microversion: 2.20 time: 0.161026 Oct 30 01:06:54.265416 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 582/1153] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:06:54 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1730 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:06:54.529722 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-a8cb261f-8b87-473a-9769-a415f14bb820 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:54.532124 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-a8cb261f-8b87-473a-9769-a415f14bb820 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:54.539106 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a8cb261f-8b87-473a-9769-a415f14bb820 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:54.539659 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a8cb261f-8b87-473a-9769-a415f14bb820 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:54.540182 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a8cb261f-8b87-473a-9769-a415f14bb820 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:54.540957 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a8cb261f-8b87-473a-9769-a415f14bb820 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:54.541607 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a8cb261f-8b87-473a-9769-a415f14bb820 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:54.542311 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a8cb261f-8b87-473a-9769-a415f14bb820 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:54.574350 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-a8cb261f-8b87-473a-9769-a415f14bb820 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049952 Oct 30 01:06:54.574926 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 583/1154] 10.222.0.16 () {58 vars in 1302 bytes} [Sun Oct 30 01:06:54 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:06:55.294008 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-8856606e-887c-4e15-a97f-049349aee22c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:55.295086 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-8856606e-887c-4e15-a97f-049349aee22c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:55.306118 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8856606e-887c-4e15-a97f-049349aee22c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:55.306929 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8856606e-887c-4e15-a97f-049349aee22c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:55.307786 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8856606e-887c-4e15-a97f-049349aee22c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:55.308503 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8856606e-887c-4e15-a97f-049349aee22c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:55.308953 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8856606e-887c-4e15-a97f-049349aee22c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:55.309373 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8856606e-887c-4e15-a97f-049349aee22c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:55.448558 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-8856606e-887c-4e15-a97f-049349aee22c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:55.461794 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-8856606e-887c-4e15-a97f-049349aee22c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:55.463986 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-8856606e-887c-4e15-a97f-049349aee22c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:55.465183 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-8856606e-887c-4e15-a97f-049349aee22c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1730 microversion: 2.20 time: 0.177999 Oct 30 01:06:55.466060 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 584/1155] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:06:55 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1730 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:06:55.592013 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-f8a518c6-fa5c-41b0-91f2-d2cf61ecef5f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:55.595504 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-f8a518c6-fa5c-41b0-91f2-d2cf61ecef5f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:55.605040 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f8a518c6-fa5c-41b0-91f2-d2cf61ecef5f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:55.605451 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f8a518c6-fa5c-41b0-91f2-d2cf61ecef5f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:55.605879 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f8a518c6-fa5c-41b0-91f2-d2cf61ecef5f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:55.606566 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f8a518c6-fa5c-41b0-91f2-d2cf61ecef5f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:55.607017 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f8a518c6-fa5c-41b0-91f2-d2cf61ecef5f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:55.607426 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f8a518c6-fa5c-41b0-91f2-d2cf61ecef5f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:55.638722 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-f8a518c6-fa5c-41b0-91f2-d2cf61ecef5f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054557 Oct 30 01:06:55.639275 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 585/1156] 10.222.0.16 () {58 vars in 1302 bytes} [Sun Oct 30 01:06:55 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:06:56.495215 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-24c5fc32-7567-4e04-bde1-a8258d1a8387 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:56.496428 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-24c5fc32-7567-4e04-bde1-a8258d1a8387 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:56.507254 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-24c5fc32-7567-4e04-bde1-a8258d1a8387 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:56.508131 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-24c5fc32-7567-4e04-bde1-a8258d1a8387 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:56.508854 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-24c5fc32-7567-4e04-bde1-a8258d1a8387 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:56.510117 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-24c5fc32-7567-4e04-bde1-a8258d1a8387 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:56.510592 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-24c5fc32-7567-4e04-bde1-a8258d1a8387 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:56.511020 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-24c5fc32-7567-4e04-bde1-a8258d1a8387 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:56.653896 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-24c5fc32-7567-4e04-bde1-a8258d1a8387 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:56.662572 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-24c5fc32-7567-4e04-bde1-a8258d1a8387 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:56.664035 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-24c5fc32-7567-4e04-bde1-a8258d1a8387 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:56.665207 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-24c5fc32-7567-4e04-bde1-a8258d1a8387 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1730 microversion: 2.20 time: 0.176664 Oct 30 01:06:56.665743 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 586/1157] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:06:56 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1730 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:06:56.670274 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-b6e4572d-77b4-45a8-8642-f47821fe155d tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:56.672482 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-b6e4572d-77b4-45a8-8642-f47821fe155d tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:56.679199 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b6e4572d-77b4-45a8-8642-f47821fe155d tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:56.679683 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b6e4572d-77b4-45a8-8642-f47821fe155d tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:56.680153 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b6e4572d-77b4-45a8-8642-f47821fe155d tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:56.680849 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b6e4572d-77b4-45a8-8642-f47821fe155d tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:56.681301 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b6e4572d-77b4-45a8-8642-f47821fe155d tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:56.681807 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b6e4572d-77b4-45a8-8642-f47821fe155d tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:56.711432 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-b6e4572d-77b4-45a8-8642-f47821fe155d tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044620 Oct 30 01:06:56.712062 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 587/1158] 10.222.0.16 () {58 vars in 1302 bytes} [Sun Oct 30 01:06:56 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:06:57.695751 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-2f2eb92c-8e89-49f5-bfd5-cbb583e22ecc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:57.696819 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-2f2eb92c-8e89-49f5-bfd5-cbb583e22ecc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:57.707560 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2f2eb92c-8e89-49f5-bfd5-cbb583e22ecc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:57.708427 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2f2eb92c-8e89-49f5-bfd5-cbb583e22ecc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:57.709161 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2f2eb92c-8e89-49f5-bfd5-cbb583e22ecc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:57.710340 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2f2eb92c-8e89-49f5-bfd5-cbb583e22ecc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:57.711135 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2f2eb92c-8e89-49f5-bfd5-cbb583e22ecc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:57.711817 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2f2eb92c-8e89-49f5-bfd5-cbb583e22ecc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:57.864425 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-2f2eb92c-8e89-49f5-bfd5-cbb583e22ecc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:57.872645 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-2f2eb92c-8e89-49f5-bfd5-cbb583e22ecc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:57.874068 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-2f2eb92c-8e89-49f5-bfd5-cbb583e22ecc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:57.875235 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-2f2eb92c-8e89-49f5-bfd5-cbb583e22ecc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1730 microversion: 2.20 time: 0.186324 Oct 30 01:06:57.875818 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 588/1159] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:06:57 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1730 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:06:57.880374 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-5b037adc-0b8e-42d7-b476-8401abe9ca50 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:57.882505 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-5b037adc-0b8e-42d7-b476-8401abe9ca50 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:57.888467 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5b037adc-0b8e-42d7-b476-8401abe9ca50 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:57.888946 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5b037adc-0b8e-42d7-b476-8401abe9ca50 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:57.889379 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5b037adc-0b8e-42d7-b476-8401abe9ca50 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:57.890060 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5b037adc-0b8e-42d7-b476-8401abe9ca50 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:57.890506 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5b037adc-0b8e-42d7-b476-8401abe9ca50 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:57.890924 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5b037adc-0b8e-42d7-b476-8401abe9ca50 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:57.919672 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-5b037adc-0b8e-42d7-b476-8401abe9ca50 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042853 Oct 30 01:06:57.920238 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 589/1160] 10.222.0.16 () {58 vars in 1302 bytes} [Sun Oct 30 01:06:57 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:06:58.903574 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-e076dbd7-e481-4ed9-82d6-d2a0ddb31981 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:58.904270 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-e076dbd7-e481-4ed9-82d6-d2a0ddb31981 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:58.911163 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e076dbd7-e481-4ed9-82d6-d2a0ddb31981 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:58.911641 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e076dbd7-e481-4ed9-82d6-d2a0ddb31981 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:58.912097 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e076dbd7-e481-4ed9-82d6-d2a0ddb31981 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:58.912788 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e076dbd7-e481-4ed9-82d6-d2a0ddb31981 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:58.913263 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e076dbd7-e481-4ed9-82d6-d2a0ddb31981 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:58.913711 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e076dbd7-e481-4ed9-82d6-d2a0ddb31981 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:59.042665 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-e076dbd7-e481-4ed9-82d6-d2a0ddb31981 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:59.051464 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-e076dbd7-e481-4ed9-82d6-d2a0ddb31981 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:59.052928 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-e076dbd7-e481-4ed9-82d6-d2a0ddb31981 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:06:59.054087 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-e076dbd7-e481-4ed9-82d6-d2a0ddb31981 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1730 microversion: 2.20 time: 0.155982 Oct 30 01:06:59.054654 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 590/1161] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:06:58 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1730 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:06:59.059299 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-46cb4823-8f23-4336-9677-778470c99c1e tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:06:59.061473 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-46cb4823-8f23-4336-9677-778470c99c1e tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:06:59.067196 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-46cb4823-8f23-4336-9677-778470c99c1e tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:59.067672 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-46cb4823-8f23-4336-9677-778470c99c1e tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:59.068133 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-46cb4823-8f23-4336-9677-778470c99c1e tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:59.068822 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-46cb4823-8f23-4336-9677-778470c99c1e tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:06:59.069269 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-46cb4823-8f23-4336-9677-778470c99c1e tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:06:59.069685 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-46cb4823-8f23-4336-9677-778470c99c1e tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:06:59.099388 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-46cb4823-8f23-4336-9677-778470c99c1e tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043783 Oct 30 01:06:59.100005 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 591/1162] 10.222.0.16 () {58 vars in 1302 bytes} [Sun Oct 30 01:06:59 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:07:00.081362 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-47b4361c-e004-4503-98e6-a5fe31c18b52 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:07:00.083009 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-47b4361c-e004-4503-98e6-a5fe31c18b52 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:07:00.091492 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-47b4361c-e004-4503-98e6-a5fe31c18b52 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:00.092079 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-47b4361c-e004-4503-98e6-a5fe31c18b52 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:00.092605 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-47b4361c-e004-4503-98e6-a5fe31c18b52 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:00.093369 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-47b4361c-e004-4503-98e6-a5fe31c18b52 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:00.093911 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-47b4361c-e004-4503-98e6-a5fe31c18b52 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:00.094394 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-47b4361c-e004-4503-98e6-a5fe31c18b52 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:00.226695 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-47b4361c-e004-4503-98e6-a5fe31c18b52 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:07:00.235404 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-47b4361c-e004-4503-98e6-a5fe31c18b52 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:07:00.236897 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-47b4361c-e004-4503-98e6-a5fe31c18b52 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:07:00.238106 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-47b4361c-e004-4503-98e6-a5fe31c18b52 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1730 microversion: 2.20 time: 0.162688 Oct 30 01:07:00.238682 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 592/1163] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:07:00 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1730 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:07:00.243274 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-845924ce-48d0-4039-ad2d-0310c3c96579 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:07:00.245474 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-845924ce-48d0-4039-ad2d-0310c3c96579 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:07:00.251517 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-845924ce-48d0-4039-ad2d-0310c3c96579 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:00.252033 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-845924ce-48d0-4039-ad2d-0310c3c96579 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:00.252466 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-845924ce-48d0-4039-ad2d-0310c3c96579 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:00.253158 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-845924ce-48d0-4039-ad2d-0310c3c96579 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:00.253606 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-845924ce-48d0-4039-ad2d-0310c3c96579 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:00.254032 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-845924ce-48d0-4039-ad2d-0310c3c96579 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:00.285522 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-845924ce-48d0-4039-ad2d-0310c3c96579 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045793 Oct 30 01:07:00.286297 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 593/1164] 10.222.0.16 () {58 vars in 1302 bytes} [Sun Oct 30 01:07:00 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:07:00.896010 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_api:SELECT=33,nova_api:UPDATE=1 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:07:00.999620 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell1:SELECT=83,nova_cell1:INSERT=4,nova_cell1:UPDATE=5 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:07:01.267636 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-a8b075bb-f779-4502-9211-fc376832da8c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:07:01.268724 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-a8b075bb-f779-4502-9211-fc376832da8c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:07:01.278594 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a8b075bb-f779-4502-9211-fc376832da8c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:01.279082 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a8b075bb-f779-4502-9211-fc376832da8c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:01.279519 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a8b075bb-f779-4502-9211-fc376832da8c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:01.280257 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a8b075bb-f779-4502-9211-fc376832da8c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:01.280708 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a8b075bb-f779-4502-9211-fc376832da8c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:01.281141 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a8b075bb-f779-4502-9211-fc376832da8c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:01.419422 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-a8b075bb-f779-4502-9211-fc376832da8c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:07:01.427437 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-a8b075bb-f779-4502-9211-fc376832da8c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:07:01.428889 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-a8b075bb-f779-4502-9211-fc376832da8c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:07:01.430051 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-a8b075bb-f779-4502-9211-fc376832da8c tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1730 microversion: 2.20 time: 0.169108 Oct 30 01:07:01.430602 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 594/1165] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:07:01 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1730 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:07:01.435540 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-d3832952-ae82-43f7-b211-64d1cb99a214 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:07:01.449651 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-d3832952-ae82-43f7-b211-64d1cb99a214 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:07:01.462175 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d3832952-ae82-43f7-b211-64d1cb99a214 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:01.462669 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d3832952-ae82-43f7-b211-64d1cb99a214 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:01.463102 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d3832952-ae82-43f7-b211-64d1cb99a214 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:01.464073 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d3832952-ae82-43f7-b211-64d1cb99a214 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:01.464530 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d3832952-ae82-43f7-b211-64d1cb99a214 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:01.465782 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d3832952-ae82-43f7-b211-64d1cb99a214 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:01.496352 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-d3832952-ae82-43f7-b211-64d1cb99a214 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.064728 Oct 30 01:07:01.496904 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 595/1166] 10.222.0.16 () {58 vars in 1302 bytes} [Sun Oct 30 01:07:01 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments => generated 197 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:07:02.459409 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-70b80205-fede-4a58-a424-998f24c6af85 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:07:02.460529 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-70b80205-fede-4a58-a424-998f24c6af85 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:07:02.471701 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-70b80205-fede-4a58-a424-998f24c6af85 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:02.472541 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-70b80205-fede-4a58-a424-998f24c6af85 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:02.473264 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-70b80205-fede-4a58-a424-998f24c6af85 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:02.474393 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-70b80205-fede-4a58-a424-998f24c6af85 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:02.475133 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-70b80205-fede-4a58-a424-998f24c6af85 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:02.475888 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-70b80205-fede-4a58-a424-998f24c6af85 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:02.480082 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell1:SELECT=100 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:07:02.611144 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-70b80205-fede-4a58-a424-998f24c6af85 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:07:02.619125 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-70b80205-fede-4a58-a424-998f24c6af85 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:07:02.620570 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-70b80205-fede-4a58-a424-998f24c6af85 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:07:02.621879 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-70b80205-fede-4a58-a424-998f24c6af85 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1730 microversion: 2.20 time: 0.169235 Oct 30 01:07:02.622474 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 596/1167] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:07:02 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1730 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:07:02.627314 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-6a3d048a-af91-4385-a367-4be09bd8d424 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:07:02.629515 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-6a3d048a-af91-4385-a367-4be09bd8d424 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:07:02.635341 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6a3d048a-af91-4385-a367-4be09bd8d424 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:02.635844 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6a3d048a-af91-4385-a367-4be09bd8d424 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:02.636273 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6a3d048a-af91-4385-a367-4be09bd8d424 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:02.637011 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6a3d048a-af91-4385-a367-4be09bd8d424 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:02.637455 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6a3d048a-af91-4385-a367-4be09bd8d424 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:02.637868 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6a3d048a-af91-4385-a367-4be09bd8d424 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:02.666674 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-6a3d048a-af91-4385-a367-4be09bd8d424 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043137 Oct 30 01:07:02.667261 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 597/1168] 10.222.0.16 () {58 vars in 1302 bytes} [Sun Oct 30 01:07:02 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:07:03.242776 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-cfc91474-235a-4d6a-a6df-f1217451f724 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "POST /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 12.357644 Oct 30 01:07:03.248869 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 572/1169] 10.222.0.16 () {60 vars in 1323 bytes} [Sun Oct 30 01:06:50 2022] POST /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a/os-volume_attachments => generated 194 bytes in 12359 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:07:03.652583 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-f4cb8960-7ea5-48d7-b6be-04a0645310dd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:07:03.653441 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-f4cb8960-7ea5-48d7-b6be-04a0645310dd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:07:03.660712 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f4cb8960-7ea5-48d7-b6be-04a0645310dd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:03.661203 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f4cb8960-7ea5-48d7-b6be-04a0645310dd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:03.661640 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f4cb8960-7ea5-48d7-b6be-04a0645310dd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:03.662358 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f4cb8960-7ea5-48d7-b6be-04a0645310dd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:03.662812 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f4cb8960-7ea5-48d7-b6be-04a0645310dd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:03.663236 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-f4cb8960-7ea5-48d7-b6be-04a0645310dd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:03.680117 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-d3c0083f-166d-4362-9cd6-02d609d43e1b tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:07:03.682559 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-d3c0083f-166d-4362-9cd6-02d609d43e1b tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:07:03.689638 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d3c0083f-166d-4362-9cd6-02d609d43e1b tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:03.690117 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d3c0083f-166d-4362-9cd6-02d609d43e1b tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:03.690549 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d3c0083f-166d-4362-9cd6-02d609d43e1b tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:03.691259 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d3c0083f-166d-4362-9cd6-02d609d43e1b tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:03.691729 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d3c0083f-166d-4362-9cd6-02d609d43e1b tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:03.692167 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d3c0083f-166d-4362-9cd6-02d609d43e1b tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:03.725873 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-d3c0083f-166d-4362-9cd6-02d609d43e1b tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048409 Oct 30 01:07:03.725873 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 573/1170] 10.222.0.16 () {58 vars in 1302 bytes} [Sun Oct 30 01:07:03 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:07:03.804456 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-f4cb8960-7ea5-48d7-b6be-04a0645310dd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:07:03.812734 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-f4cb8960-7ea5-48d7-b6be-04a0645310dd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:07:03.814167 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-f4cb8960-7ea5-48d7-b6be-04a0645310dd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:07:03.815537 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-f4cb8960-7ea5-48d7-b6be-04a0645310dd tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1730 microversion: 2.20 time: 0.169783 Oct 30 01:07:03.816367 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 598/1171] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:07:03 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1730 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:07:04.741758 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-e0e96957-9d2c-4581-8f9a-6e0f3c561f90 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:07:04.746959 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-e0e96957-9d2c-4581-8f9a-6e0f3c561f90 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:07:04.758113 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e0e96957-9d2c-4581-8f9a-6e0f3c561f90 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:04.758910 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e0e96957-9d2c-4581-8f9a-6e0f3c561f90 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:04.759635 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e0e96957-9d2c-4581-8f9a-6e0f3c561f90 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:04.760899 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e0e96957-9d2c-4581-8f9a-6e0f3c561f90 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:04.761643 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e0e96957-9d2c-4581-8f9a-6e0f3c561f90 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:04.762347 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e0e96957-9d2c-4581-8f9a-6e0f3c561f90 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:04.792269 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-e0e96957-9d2c-4581-8f9a-6e0f3c561f90 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057547 Oct 30 01:07:04.792819 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 574/1172] 10.222.0.16 () {58 vars in 1302 bytes} [Sun Oct 30 01:07:04 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:07:04.842182 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-d97b2961-cb41-480e-afff-8216f8bd1134 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:07:04.843136 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-d97b2961-cb41-480e-afff-8216f8bd1134 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:07:04.849890 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d97b2961-cb41-480e-afff-8216f8bd1134 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:04.850379 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d97b2961-cb41-480e-afff-8216f8bd1134 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:04.850813 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d97b2961-cb41-480e-afff-8216f8bd1134 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:04.851521 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d97b2961-cb41-480e-afff-8216f8bd1134 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:04.852008 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d97b2961-cb41-480e-afff-8216f8bd1134 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:04.852425 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d97b2961-cb41-480e-afff-8216f8bd1134 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:04.988885 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-d97b2961-cb41-480e-afff-8216f8bd1134 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:07:04.997258 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-d97b2961-cb41-480e-afff-8216f8bd1134 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:07:04.998696 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-d97b2961-cb41-480e-afff-8216f8bd1134 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:07:04.999929 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-d97b2961-cb41-480e-afff-8216f8bd1134 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1730 microversion: 2.20 time: 0.162109 Oct 30 01:07:05.000503 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 599/1173] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:07:04 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1730 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:07:05.809661 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-86c76ecb-8afb-4b32-99a0-37ebcb67ffa8 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:07:05.812127 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-86c76ecb-8afb-4b32-99a0-37ebcb67ffa8 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:07:05.818983 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-86c76ecb-8afb-4b32-99a0-37ebcb67ffa8 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:05.819457 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-86c76ecb-8afb-4b32-99a0-37ebcb67ffa8 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:05.819922 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-86c76ecb-8afb-4b32-99a0-37ebcb67ffa8 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:05.820621 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-86c76ecb-8afb-4b32-99a0-37ebcb67ffa8 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:05.821064 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-86c76ecb-8afb-4b32-99a0-37ebcb67ffa8 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:05.821478 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-86c76ecb-8afb-4b32-99a0-37ebcb67ffa8 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:05.851397 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-86c76ecb-8afb-4b32-99a0-37ebcb67ffa8 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048608 Oct 30 01:07:05.851981 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 575/1174] 10.222.0.16 () {58 vars in 1302 bytes} [Sun Oct 30 01:07:05 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:07:06.026314 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-3839ec7f-641c-45be-91e1-b6dc44b2cf40 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:07:06.026926 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-3839ec7f-641c-45be-91e1-b6dc44b2cf40 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:07:06.029448 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_api:SELECT=80 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:07:06.034787 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3839ec7f-641c-45be-91e1-b6dc44b2cf40 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:06.035274 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3839ec7f-641c-45be-91e1-b6dc44b2cf40 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:06.035710 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3839ec7f-641c-45be-91e1-b6dc44b2cf40 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:06.036431 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3839ec7f-641c-45be-91e1-b6dc44b2cf40 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:06.036881 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3839ec7f-641c-45be-91e1-b6dc44b2cf40 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:06.037315 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3839ec7f-641c-45be-91e1-b6dc44b2cf40 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:06.175825 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-3839ec7f-641c-45be-91e1-b6dc44b2cf40 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:07:06.188161 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-3839ec7f-641c-45be-91e1-b6dc44b2cf40 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:07:06.190773 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-3839ec7f-641c-45be-91e1-b6dc44b2cf40 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:07:06.192094 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-3839ec7f-641c-45be-91e1-b6dc44b2cf40 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1730 microversion: 2.20 time: 0.169779 Oct 30 01:07:06.192751 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 600/1175] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:07:06 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1730 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:07:06.548144 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [req-73a2f256-5abe-4ab0-b52f-807d3e37afc7 req-3ae551f7-4c42-4b2f-90a9-268181d210bb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f4033e06-e2d3-4abb-ba25-eddfaf3e7e20", "name": "network-vif-plugged", "status": "completed", "tag": "e5d64f79-b32b-48ac-8231-fbd53262a835"}]} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:07:06.558955 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-73a2f256-5abe-4ab0-b52f-807d3e37afc7 req-3ae551f7-4c42-4b2f-90a9-268181d210bb service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:06.559436 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-73a2f256-5abe-4ab0-b52f-807d3e37afc7 req-3ae551f7-4c42-4b2f-90a9-268181d210bb service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:06.560004 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-73a2f256-5abe-4ab0-b52f-807d3e37afc7 req-3ae551f7-4c42-4b2f-90a9-268181d210bb service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:06.572195 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.compute.server_external_events [req-73a2f256-5abe-4ab0-b52f-807d3e37afc7 req-3ae551f7-4c42-4b2f-90a9-268181d210bb service nova] Creating event network-vif-plugged:e5d64f79-b32b-48ac-8231-fbd53262a835 for instance f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 on hv-cinder-maste Oct 30 01:07:06.577973 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [req-73a2f256-5abe-4ab0-b52f-807d3e37afc7 req-3ae551f7-4c42-4b2f-90a9-268181d210bb service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033978 Oct 30 01:07:06.578579 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 576/1176] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:07:06 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 30 01:07:06.863955 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-11f0fadb-b4ed-4943-8100-76a659c396ad tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:07:06.866248 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-11f0fadb-b4ed-4943-8100-76a659c396ad tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:07:06.872575 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-11f0fadb-b4ed-4943-8100-76a659c396ad tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:06.873090 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-11f0fadb-b4ed-4943-8100-76a659c396ad tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:06.873519 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-11f0fadb-b4ed-4943-8100-76a659c396ad tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:06.874214 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-11f0fadb-b4ed-4943-8100-76a659c396ad tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:06.874657 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-11f0fadb-b4ed-4943-8100-76a659c396ad tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:06.875081 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-11f0fadb-b4ed-4943-8100-76a659c396ad tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:06.904919 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-11f0fadb-b4ed-4943-8100-76a659c396ad tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045371 Oct 30 01:07:06.905458 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 601/1177] 10.222.0.16 () {58 vars in 1302 bytes} [Sun Oct 30 01:07:06 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:07:07.219821 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-205a7ae1-9dec-4127-b902-bb5ee5c7bffe tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:07:07.221136 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-205a7ae1-9dec-4127-b902-bb5ee5c7bffe tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:07:07.227453 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-205a7ae1-9dec-4127-b902-bb5ee5c7bffe tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:07.227966 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-205a7ae1-9dec-4127-b902-bb5ee5c7bffe tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:07.228405 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-205a7ae1-9dec-4127-b902-bb5ee5c7bffe tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:07.229100 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-205a7ae1-9dec-4127-b902-bb5ee5c7bffe tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:07.229546 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-205a7ae1-9dec-4127-b902-bb5ee5c7bffe tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:07.229963 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-205a7ae1-9dec-4127-b902-bb5ee5c7bffe tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:07.366586 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-205a7ae1-9dec-4127-b902-bb5ee5c7bffe tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:07:07.375119 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-205a7ae1-9dec-4127-b902-bb5ee5c7bffe tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:07:07.376668 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-205a7ae1-9dec-4127-b902-bb5ee5c7bffe tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:07:07.377899 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-205a7ae1-9dec-4127-b902-bb5ee5c7bffe tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1730 microversion: 2.20 time: 0.163751 Oct 30 01:07:07.378757 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 577/1178] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:07:07 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1730 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:07:07.918375 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-91039507-7c12-4e9d-a9d8-595ca485753f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:07:07.920416 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-91039507-7c12-4e9d-a9d8-595ca485753f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:07:07.926481 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-91039507-7c12-4e9d-a9d8-595ca485753f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:07.926969 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-91039507-7c12-4e9d-a9d8-595ca485753f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:07.927368 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-91039507-7c12-4e9d-a9d8-595ca485753f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:07.928092 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-91039507-7c12-4e9d-a9d8-595ca485753f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:07.928534 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-91039507-7c12-4e9d-a9d8-595ca485753f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:07.928949 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-91039507-7c12-4e9d-a9d8-595ca485753f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:07.959846 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-91039507-7c12-4e9d-a9d8-595ca485753f tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046023 Oct 30 01:07:07.960621 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 602/1179] 10.222.0.16 () {58 vars in 1302 bytes} [Sun Oct 30 01:07:07 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:07:08.402968 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-1b5e6710-8f69-4d15-beed-89b2d19021a2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:07:08.403565 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-1b5e6710-8f69-4d15-beed-89b2d19021a2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:07:08.410912 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1b5e6710-8f69-4d15-beed-89b2d19021a2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:08.411527 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1b5e6710-8f69-4d15-beed-89b2d19021a2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:08.412247 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1b5e6710-8f69-4d15-beed-89b2d19021a2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:08.412957 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1b5e6710-8f69-4d15-beed-89b2d19021a2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:08.413407 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1b5e6710-8f69-4d15-beed-89b2d19021a2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:08.413848 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1b5e6710-8f69-4d15-beed-89b2d19021a2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:08.554240 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-1b5e6710-8f69-4d15-beed-89b2d19021a2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:07:08.562607 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-1b5e6710-8f69-4d15-beed-89b2d19021a2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:07:08.564067 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-1b5e6710-8f69-4d15-beed-89b2d19021a2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:07:08.565290 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-1b5e6710-8f69-4d15-beed-89b2d19021a2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1730 microversion: 2.20 time: 0.166426 Oct 30 01:07:08.565832 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 578/1180] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:07:08 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1730 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:07:08.973151 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-d0de2e8e-c658-4d2d-a673-21caf10f0e23 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:07:08.976006 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-d0de2e8e-c658-4d2d-a673-21caf10f0e23 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:07:08.987000 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d0de2e8e-c658-4d2d-a673-21caf10f0e23 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:08.987399 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d0de2e8e-c658-4d2d-a673-21caf10f0e23 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:08.987884 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d0de2e8e-c658-4d2d-a673-21caf10f0e23 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:08.988645 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d0de2e8e-c658-4d2d-a673-21caf10f0e23 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:08.989133 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d0de2e8e-c658-4d2d-a673-21caf10f0e23 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:08.989593 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d0de2e8e-c658-4d2d-a673-21caf10f0e23 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:09.032825 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-d0de2e8e-c658-4d2d-a673-21caf10f0e23 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.064184 Oct 30 01:07:09.033593 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 603/1181] 10.222.0.16 () {58 vars in 1302 bytes} [Sun Oct 30 01:07:08 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments => generated 197 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:07:09.591017 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-282a9428-633b-46df-bb90-bafe80bb85fc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:07:09.592541 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-282a9428-633b-46df-bb90-bafe80bb85fc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:07:09.598967 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-282a9428-633b-46df-bb90-bafe80bb85fc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:09.599535 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-282a9428-633b-46df-bb90-bafe80bb85fc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:09.600048 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-282a9428-633b-46df-bb90-bafe80bb85fc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:09.600856 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-282a9428-633b-46df-bb90-bafe80bb85fc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:09.601397 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-282a9428-633b-46df-bb90-bafe80bb85fc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:09.601941 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-282a9428-633b-46df-bb90-bafe80bb85fc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:09.780193 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-282a9428-633b-46df-bb90-bafe80bb85fc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:07:09.790740 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-282a9428-633b-46df-bb90-bafe80bb85fc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:07:09.792534 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-282a9428-633b-46df-bb90-bafe80bb85fc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:07:09.793893 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-282a9428-633b-46df-bb90-bafe80bb85fc tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1730 microversion: 2.20 time: 0.207367 Oct 30 01:07:09.795207 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 579/1182] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:07:09 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1730 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:07:10.046335 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-61d4acfc-5e85-44da-abd5-48695a49fc68 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:07:10.048605 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-61d4acfc-5e85-44da-abd5-48695a49fc68 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:07:10.065547 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-61d4acfc-5e85-44da-abd5-48695a49fc68 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:10.066747 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-61d4acfc-5e85-44da-abd5-48695a49fc68 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:10.067361 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-61d4acfc-5e85-44da-abd5-48695a49fc68 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:10.068371 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-61d4acfc-5e85-44da-abd5-48695a49fc68 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:10.069320 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-61d4acfc-5e85-44da-abd5-48695a49fc68 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:10.069924 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-61d4acfc-5e85-44da-abd5-48695a49fc68 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:10.111201 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-61d4acfc-5e85-44da-abd5-48695a49fc68 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.069314 Oct 30 01:07:10.112235 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 604/1183] 10.222.0.16 () {58 vars in 1302 bytes} [Sun Oct 30 01:07:10 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments => generated 25 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 30 01:07:10.123634 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-097b10d6-bfb1-4467-82af-2a167893f8d0 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:07:10.124171 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-097b10d6-bfb1-4467-82af-2a167893f8d0 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:07:10.130550 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-097b10d6-bfb1-4467-82af-2a167893f8d0 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:10.131307 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-097b10d6-bfb1-4467-82af-2a167893f8d0 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:10.131862 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-097b10d6-bfb1-4467-82af-2a167893f8d0 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:10.132584 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-097b10d6-bfb1-4467-82af-2a167893f8d0 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:10.133036 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-097b10d6-bfb1-4467-82af-2a167893f8d0 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:10.133660 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-097b10d6-bfb1-4467-82af-2a167893f8d0 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:10.221199 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.wsgi [None req-097b10d6-bfb1-4467-82af-2a167893f8d0 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] HTTP exception thrown: Instance 6f5485fd-b825-45e7-a5be-eb96208c09b7 is not attached to volume a125d045-591a-4811-a456-a51a7989295e Oct 30 01:07:10.224089 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-097b10d6-bfb1-4467-82af-2a167893f8d0 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Request method failure captured: Oct 30 01:07:10.224089 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: request: DELETE /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments/a125d045-591a-4811-a456-a51a7989295e HTTP/1.1 Oct 30 01:07:10.224089 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept: application/json Oct 30 01:07:10.224089 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept-Encoding: identity Oct 30 01:07:10.224089 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Connection: close Oct 30 01:07:10.224089 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Length: 0 Oct 30 01:07:10.224089 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Type: application/json Oct 30 01:07:10.224089 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Host: 10.222.0.16 Oct 30 01:07:10.224089 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Openstack-System-Scope: None Oct 30 01:07:10.224089 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: User-Agent: python-urllib3/1.26.12 Oct 30 01:07:10.224089 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Auth-Token: gAAAAABjXc02LpoTvIxCzkoArpoYgkyqzW1tM35LLf5rByOvZIZILCO9QtJeuOUgW2KT-dHtq5aE3v3Rprc5-tW1eSjg7X8kBJZ6YQV0VC0BUzpkNcehgX959TNTIn_nYAvRYeikCqC4Kl-YoiOfiz62QUy7tQ9_1gRKFNapCAPDKGUN0o9pzyY Oct 30 01:07:10.224089 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Id: None Oct 30 01:07:10.224089 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Name: None Oct 30 01:07:10.224089 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Identity-Status: Confirmed Oct 30 01:07:10.224089 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Is-Admin-Project: True Oct 30 01:07:10.224089 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Id: default Oct 30 01:07:10.224089 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Name: Default Oct 30 01:07:10.224089 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Id: 1dcbf2e9fb2e4a4698633fcbb07c30cd Oct 30 01:07:10.224089 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Name: tempest-TestMinimumBasicScenario-1933647695 Oct 30 01:07:10.224089 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Role: reader,member Oct 30 01:07:10.224089 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Roles: reader,member Oct 30 01:07:10.225709 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_1dcbf2e9fb2e4a4698633fcbb07c30cd"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1dcbf2e9fb2e4a4698633fcbb07c30cd"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/1dcbf2e9fb2e4a4698633fcbb07c30cd"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1dcbf2e9fb2e4a4698633fcbb07c30cd"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 01:07:10.225709 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant: tempest-TestMinimumBasicScenario-1933647695 Oct 30 01:07:10.225709 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Id: 1dcbf2e9fb2e4a4698633fcbb07c30cd Oct 30 01:07:10.225709 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Name: tempest-TestMinimumBasicScenario-1933647695 Oct 30 01:07:10.225709 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User: tempest-TestMinimumBasicScenario-1933647695-project Oct 30 01:07:10.225709 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Id: default Oct 30 01:07:10.225709 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Name: Default Oct 30 01:07:10.225709 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Id: 05eed70effab4e38ae469ab44d1114db Oct 30 01:07:10.225709 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Name: tempest-TestMinimumBasicScenario-1933647695-project Oct 30 01:07:10.225709 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: method: > Oct 30 01:07:10.225709 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: exception: Instance 6f5485fd-b825-45e7-a5be-eb96208c09b7 is not attached to volume a125d045-591a-4811-a456-a51a7989295e Oct 30 01:07:10.226455 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: action_args: {'server_id': '6f5485fd-b825-45e7-a5be-eb96208c09b7', 'id': 'a125d045-591a-4811-a456-a51a7989295e'} Oct 30 01:07:10.226455 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 01:07:10.226455 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:07:10.226455 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Oct 30 01:07:10.226455 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Oct 30 01:07:10.226455 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 30 01:07:10.226455 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 30 01:07:10.226455 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Oct 30 01:07:10.226455 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Oct 30 01:07:10.226455 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id a125d045-591a-4811-a456-a51a7989295e. Oct 30 01:07:10.226455 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:07:10.226455 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:07:10.226455 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:07:10.226455 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:07:10.226455 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:07:10.226455 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:07:10.226455 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 01:07:10.226455 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 01:07:10.226455 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 01:07:10.228119 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 01:07:10.228119 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Oct 30 01:07:10.228119 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Oct 30 01:07:10.228119 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6f5485fd-b825-45e7-a5be-eb96208c09b7 is not attached to volume a125d045-591a-4811-a456-a51a7989295e Oct 30 01:07:10.228119 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:07:10.228119 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:07:10.228119 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:07:10.228119 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:07:10.228119 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:07:10.228119 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:07:10.228119 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 01:07:10.228119 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 01:07:10.228119 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6f5485fd-b825-45e7-a5be-eb96208c09b7 is not attached to volume a125d045-591a-4811-a456-a51a7989295e Oct 30 01:07:10.228119 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:07:10.228119 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-097b10d6-bfb1-4467-82af-2a167893f8d0 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Returning 404 to user: Instance 6f5485fd-b825-45e7-a5be-eb96208c09b7 is not attached to volume a125d045-591a-4811-a456-a51a7989295e {{(pid=118962) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 01:07:10.229145 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-097b10d6-bfb1-4467-82af-2a167893f8d0 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "DELETE /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments/a125d045-591a-4811-a456-a51a7989295e" status: 404 len: 154 microversion: 2.1 time: 0.106202 Oct 30 01:07:10.229145 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 580/1184] 10.222.0.16 () {58 vars in 1416 bytes} [Sun Oct 30 01:07:10 2022] DELETE /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7/os-volume_attachments/a125d045-591a-4811-a456-a51a7989295e => generated 154 bytes in 107 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 30 01:07:10.835531 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-cdc797fb-5908-4c7a-8cb1-920a6e1e72bb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:07:10.836104 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-cdc797fb-5908-4c7a-8cb1-920a6e1e72bb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:07:10.843429 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cdc797fb-5908-4c7a-8cb1-920a6e1e72bb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:10.843983 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cdc797fb-5908-4c7a-8cb1-920a6e1e72bb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:10.844424 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cdc797fb-5908-4c7a-8cb1-920a6e1e72bb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:10.845157 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cdc797fb-5908-4c7a-8cb1-920a6e1e72bb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:10.845610 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cdc797fb-5908-4c7a-8cb1-920a6e1e72bb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:10.846037 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cdc797fb-5908-4c7a-8cb1-920a6e1e72bb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:10.999527 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-cdc797fb-5908-4c7a-8cb1-920a6e1e72bb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:07:11.009412 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-cdc797fb-5908-4c7a-8cb1-920a6e1e72bb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:07:11.011110 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-cdc797fb-5908-4c7a-8cb1-920a6e1e72bb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:07:11.012700 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-cdc797fb-5908-4c7a-8cb1-920a6e1e72bb tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1730 microversion: 2.20 time: 0.181753 Oct 30 01:07:11.013761 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 605/1185] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:07:10 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1730 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:07:11.397639 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-8ae2f7b0-e75b-47e1-be4c-1e22c7645aac tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:07:11.398137 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-8ae2f7b0-e75b-47e1-be4c-1e22c7645aac tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:07:11.404885 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8ae2f7b0-e75b-47e1-be4c-1e22c7645aac tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:11.405359 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8ae2f7b0-e75b-47e1-be4c-1e22c7645aac tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:11.406006 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8ae2f7b0-e75b-47e1-be4c-1e22c7645aac tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:11.406707 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8ae2f7b0-e75b-47e1-be4c-1e22c7645aac tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:11.407147 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8ae2f7b0-e75b-47e1-be4c-1e22c7645aac tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:11.407553 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8ae2f7b0-e75b-47e1-be4c-1e22c7645aac tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:11.434100 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-8ae2f7b0-e75b-47e1-be4c-1e22c7645aac tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Going to try to terminate instance {{(pid=118962) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 30 01:07:11.545985 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-8ae2f7b0-e75b-47e1-be4c-1e22c7645aac tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "DELETE /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7" status: 204 len: 0 microversion: 2.1 time: 0.151774 Oct 30 01:07:11.547334 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 581/1186] 10.222.0.16 () {58 vars in 1239 bytes} [Sun Oct 30 01:07:11 2022] DELETE /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 30 01:07:11.557832 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-876ae86c-93da-4f5f-8e20-43882147cdf3 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:07:11.559101 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-876ae86c-93da-4f5f-8e20-43882147cdf3 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:07:11.569876 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-876ae86c-93da-4f5f-8e20-43882147cdf3 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:11.570478 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-876ae86c-93da-4f5f-8e20-43882147cdf3 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:11.571099 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-876ae86c-93da-4f5f-8e20-43882147cdf3 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:11.572139 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-876ae86c-93da-4f5f-8e20-43882147cdf3 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:11.572719 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-876ae86c-93da-4f5f-8e20-43882147cdf3 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:11.573298 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-876ae86c-93da-4f5f-8e20-43882147cdf3 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:11.867678 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-876ae86c-93da-4f5f-8e20-43882147cdf3 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '05eed70effab4e38ae469ab44d1114db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dcbf2e9fb2e4a4698633fcbb07c30cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:07:11.886657 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-876ae86c-93da-4f5f-8e20-43882147cdf3 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7" status: 200 len: 1555 microversion: 2.1 time: 0.335027 Oct 30 01:07:11.887240 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 606/1187] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:07:11 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7 => generated 1555 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 01:07:12.039353 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-9d459a8e-3042-4b8e-bb0f-7464502010bf tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:07:12.040074 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-9d459a8e-3042-4b8e-bb0f-7464502010bf tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:07:12.052617 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9d459a8e-3042-4b8e-bb0f-7464502010bf tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:12.053580 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9d459a8e-3042-4b8e-bb0f-7464502010bf tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:12.054255 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9d459a8e-3042-4b8e-bb0f-7464502010bf tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:12.055140 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9d459a8e-3042-4b8e-bb0f-7464502010bf tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:12.055588 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9d459a8e-3042-4b8e-bb0f-7464502010bf tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:12.056030 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-9d459a8e-3042-4b8e-bb0f-7464502010bf tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:12.312960 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-9d459a8e-3042-4b8e-bb0f-7464502010bf tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:07:12.327122 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-9d459a8e-3042-4b8e-bb0f-7464502010bf tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:07:12.331353 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-9d459a8e-3042-4b8e-bb0f-7464502010bf tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:07:12.333548 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-9d459a8e-3042-4b8e-bb0f-7464502010bf tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1730 microversion: 2.20 time: 0.298367 Oct 30 01:07:12.334657 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 582/1188] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:07:12 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1730 bytes in 301 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:07:12.910895 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-6fcbd87e-2dc9-46fe-a8c0-4912c3f2f23a tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:07:12.911567 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-6fcbd87e-2dc9-46fe-a8c0-4912c3f2f23a tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:07:12.920033 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6fcbd87e-2dc9-46fe-a8c0-4912c3f2f23a tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:12.920455 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6fcbd87e-2dc9-46fe-a8c0-4912c3f2f23a tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:12.921373 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6fcbd87e-2dc9-46fe-a8c0-4912c3f2f23a tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:12.922090 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6fcbd87e-2dc9-46fe-a8c0-4912c3f2f23a tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:12.922539 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6fcbd87e-2dc9-46fe-a8c0-4912c3f2f23a tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:12.923479 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6fcbd87e-2dc9-46fe-a8c0-4912c3f2f23a tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:13.133305 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-6fcbd87e-2dc9-46fe-a8c0-4912c3f2f23a tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '05eed70effab4e38ae469ab44d1114db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dcbf2e9fb2e4a4698633fcbb07c30cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:07:13.142010 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-6fcbd87e-2dc9-46fe-a8c0-4912c3f2f23a tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7" status: 200 len: 1555 microversion: 2.1 time: 0.235227 Oct 30 01:07:13.142596 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 607/1189] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:07:12 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7 => generated 1555 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:07:13.361232 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-029ec2d6-ccff-43d5-aba3-112558067185 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:07:13.362963 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-029ec2d6-ccff-43d5-aba3-112558067185 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:07:13.374703 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-029ec2d6-ccff-43d5-aba3-112558067185 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:13.375230 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-029ec2d6-ccff-43d5-aba3-112558067185 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:13.375986 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-029ec2d6-ccff-43d5-aba3-112558067185 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:13.384187 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-029ec2d6-ccff-43d5-aba3-112558067185 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:13.385022 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-029ec2d6-ccff-43d5-aba3-112558067185 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:13.387985 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-029ec2d6-ccff-43d5-aba3-112558067185 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:13.879481 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-029ec2d6-ccff-43d5-aba3-112558067185 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:07:13.888439 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-029ec2d6-ccff-43d5-aba3-112558067185 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:07:13.890415 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-029ec2d6-ccff-43d5-aba3-112558067185 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:07:13.891882 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-029ec2d6-ccff-43d5-aba3-112558067185 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1717 microversion: 2.20 time: 0.534535 Oct 30 01:07:13.892643 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 583/1190] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:07:13 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1717 bytes in 536 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:07:14.166004 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-014f5c65-6970-4bbf-a4a4-01a867d17b3c tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:07:14.169569 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-014f5c65-6970-4bbf-a4a4-01a867d17b3c tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:07:14.178963 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-014f5c65-6970-4bbf-a4a4-01a867d17b3c tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:14.179860 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-014f5c65-6970-4bbf-a4a4-01a867d17b3c tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:14.180515 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-014f5c65-6970-4bbf-a4a4-01a867d17b3c tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:14.187644 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-014f5c65-6970-4bbf-a4a4-01a867d17b3c tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:14.188478 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-014f5c65-6970-4bbf-a4a4-01a867d17b3c tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:14.189107 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-014f5c65-6970-4bbf-a4a4-01a867d17b3c tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:14.336773 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-014f5c65-6970-4bbf-a4a4-01a867d17b3c tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '05eed70effab4e38ae469ab44d1114db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dcbf2e9fb2e4a4698633fcbb07c30cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:07:14.346989 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-014f5c65-6970-4bbf-a4a4-01a867d17b3c tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7" status: 200 len: 1555 microversion: 2.1 time: 0.184874 Oct 30 01:07:14.359809 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 608/1191] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:07:14 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7 => generated 1555 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:07:14.867487 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [req-c04aa5b3-ea9c-4328-a20a-e4092054a3ac req-a1b370c8-cf2a-4ae5-8440-85e14fe0e6d4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6f5485fd-b825-45e7-a5be-eb96208c09b7", "name": "network-vif-deleted", "tag": "ad10e942-b527-4fbd-867f-e161a78a11bc"}]} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:07:14.878331 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-c04aa5b3-ea9c-4328-a20a-e4092054a3ac req-a1b370c8-cf2a-4ae5-8440-85e14fe0e6d4 service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:14.878840 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-c04aa5b3-ea9c-4328-a20a-e4092054a3ac req-a1b370c8-cf2a-4ae5-8440-85e14fe0e6d4 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:14.879266 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-c04aa5b3-ea9c-4328-a20a-e4092054a3ac req-a1b370c8-cf2a-4ae5-8440-85e14fe0e6d4 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:14.901456 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.compute.server_external_events [req-c04aa5b3-ea9c-4328-a20a-e4092054a3ac req-a1b370c8-cf2a-4ae5-8440-85e14fe0e6d4 service nova] Creating event network-vif-deleted:ad10e942-b527-4fbd-867f-e161a78a11bc for instance 6f5485fd-b825-45e7-a5be-eb96208c09b7 on hv-cinder-maste Oct 30 01:07:14.909692 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [req-c04aa5b3-ea9c-4328-a20a-e4092054a3ac req-a1b370c8-cf2a-4ae5-8440-85e14fe0e6d4 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.291582 Oct 30 01:07:14.911154 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 584/1192] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:07:14 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:07:15.372577 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-35d086ab-3e11-45a0-9b1b-7fb91670552e tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:07:15.373381 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-35d086ab-3e11-45a0-9b1b-7fb91670552e tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:07:15.381218 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-35d086ab-3e11-45a0-9b1b-7fb91670552e tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:15.382096 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-35d086ab-3e11-45a0-9b1b-7fb91670552e tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:15.382532 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-35d086ab-3e11-45a0-9b1b-7fb91670552e tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:15.383259 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-35d086ab-3e11-45a0-9b1b-7fb91670552e tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:15.383743 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-35d086ab-3e11-45a0-9b1b-7fb91670552e tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:15.384169 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-35d086ab-3e11-45a0-9b1b-7fb91670552e tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:15.419357 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-35d086ab-3e11-45a0-9b1b-7fb91670552e tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:15.419900 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-35d086ab-3e11-45a0-9b1b-7fb91670552e tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:15.420338 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-35d086ab-3e11-45a0-9b1b-7fb91670552e tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:15.420789 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.objects.instance [None req-35d086ab-3e11-45a0-9b1b-7fb91670552e tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lazy-loading 'fault' on Instance uuid 6f5485fd-b825-45e7-a5be-eb96208c09b7 {{(pid=118961) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 30 01:07:15.482445 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-35d086ab-3e11-45a0-9b1b-7fb91670552e tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '05eed70effab4e38ae469ab44d1114db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dcbf2e9fb2e4a4698633fcbb07c30cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:07:15.491976 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-35d086ab-3e11-45a0-9b1b-7fb91670552e tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7" status: 200 len: 1348 microversion: 2.1 time: 0.123301 Oct 30 01:07:15.492938 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 609/1193] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:07:15 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7 => generated 1348 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:07:16.517355 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-e02a0a6d-959e-450f-bda9-3679d59c9fa7 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:07:16.518210 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-e02a0a6d-959e-450f-bda9-3679d59c9fa7 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] [instance: 6f5485fd-b825-45e7-a5be-eb96208c09b7] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:07:16.524977 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e02a0a6d-959e-450f-bda9-3679d59c9fa7 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:16.525531 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e02a0a6d-959e-450f-bda9-3679d59c9fa7 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:16.526159 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e02a0a6d-959e-450f-bda9-3679d59c9fa7 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:16.527069 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e02a0a6d-959e-450f-bda9-3679d59c9fa7 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:16.527701 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e02a0a6d-959e-450f-bda9-3679d59c9fa7 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:16.528420 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e02a0a6d-959e-450f-bda9-3679d59c9fa7 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:16.546751 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.wsgi [None req-e02a0a6d-959e-450f-bda9-3679d59c9fa7 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] HTTP exception thrown: Instance 6f5485fd-b825-45e7-a5be-eb96208c09b7 could not be found. Oct 30 01:07:16.550843 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-e02a0a6d-959e-450f-bda9-3679d59c9fa7 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Request method failure captured: Oct 30 01:07:16.550843 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: request: GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7 HTTP/1.1 Oct 30 01:07:16.550843 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept: application/json Oct 30 01:07:16.550843 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept-Encoding: identity Oct 30 01:07:16.550843 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Connection: close Oct 30 01:07:16.550843 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Length: 0 Oct 30 01:07:16.550843 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Type: application/json Oct 30 01:07:16.550843 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Host: 10.222.0.16 Oct 30 01:07:16.550843 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Openstack-System-Scope: None Oct 30 01:07:16.550843 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: User-Agent: python-urllib3/1.26.12 Oct 30 01:07:16.550843 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Auth-Token: gAAAAABjXc02LpoTvIxCzkoArpoYgkyqzW1tM35LLf5rByOvZIZILCO9QtJeuOUgW2KT-dHtq5aE3v3Rprc5-tW1eSjg7X8kBJZ6YQV0VC0BUzpkNcehgX959TNTIn_nYAvRYeikCqC4Kl-YoiOfiz62QUy7tQ9_1gRKFNapCAPDKGUN0o9pzyY Oct 30 01:07:16.550843 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Id: None Oct 30 01:07:16.550843 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Name: None Oct 30 01:07:16.550843 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Identity-Status: Confirmed Oct 30 01:07:16.550843 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Is-Admin-Project: True Oct 30 01:07:16.550843 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Id: default Oct 30 01:07:16.550843 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Name: Default Oct 30 01:07:16.550843 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Id: 1dcbf2e9fb2e4a4698633fcbb07c30cd Oct 30 01:07:16.550843 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Name: tempest-TestMinimumBasicScenario-1933647695 Oct 30 01:07:16.550843 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Role: reader,member Oct 30 01:07:16.550843 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Roles: reader,member Oct 30 01:07:16.552637 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_1dcbf2e9fb2e4a4698633fcbb07c30cd"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1dcbf2e9fb2e4a4698633fcbb07c30cd"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/1dcbf2e9fb2e4a4698633fcbb07c30cd"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1dcbf2e9fb2e4a4698633fcbb07c30cd"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 01:07:16.552637 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant: tempest-TestMinimumBasicScenario-1933647695 Oct 30 01:07:16.552637 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Id: 1dcbf2e9fb2e4a4698633fcbb07c30cd Oct 30 01:07:16.552637 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Name: tempest-TestMinimumBasicScenario-1933647695 Oct 30 01:07:16.552637 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User: tempest-TestMinimumBasicScenario-1933647695-project Oct 30 01:07:16.552637 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Id: default Oct 30 01:07:16.552637 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Name: Default Oct 30 01:07:16.552637 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Id: 05eed70effab4e38ae469ab44d1114db Oct 30 01:07:16.552637 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Name: tempest-TestMinimumBasicScenario-1933647695-project Oct 30 01:07:16.552637 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: method: > Oct 30 01:07:16.552637 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: exception: Instance 6f5485fd-b825-45e7-a5be-eb96208c09b7 could not be found. Oct 30 01:07:16.552637 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: action_args: {'id': '6f5485fd-b825-45e7-a5be-eb96208c09b7'} Oct 30 01:07:16.553496 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 01:07:16.553496 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:07:16.553496 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 30 01:07:16.553496 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 30 01:07:16.553496 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 30 01:07:16.553496 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 30 01:07:16.553496 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 30 01:07:16.553496 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 30 01:07:16.553496 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 30 01:07:16.553496 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 30 01:07:16.553496 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6f5485fd-b825-45e7-a5be-eb96208c09b7 could not be found. Oct 30 01:07:16.553496 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:07:16.553496 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:07:16.553496 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:07:16.553496 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:07:16.553496 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:07:16.553496 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:07:16.553496 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 01:07:16.553496 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 01:07:16.553496 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 01:07:16.554727 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 01:07:16.554727 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 30 01:07:16.554727 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 30 01:07:16.554727 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 30 01:07:16.554727 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 30 01:07:16.554727 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 30 01:07:16.554727 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 30 01:07:16.554727 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6f5485fd-b825-45e7-a5be-eb96208c09b7 could not be found. Oct 30 01:07:16.554727 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:07:16.554727 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:07:16.554727 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:07:16.554727 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:07:16.554727 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:07:16.554727 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:07:16.554727 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 01:07:16.554727 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 01:07:16.554727 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6f5485fd-b825-45e7-a5be-eb96208c09b7 could not be found. Oct 30 01:07:16.554727 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:07:16.555897 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-e02a0a6d-959e-450f-bda9-3679d59c9fa7 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Returning 404 to user: Instance 6f5485fd-b825-45e7-a5be-eb96208c09b7 could not be found. {{(pid=118962) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 01:07:16.555897 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-e02a0a6d-959e-450f-bda9-3679d59c9fa7 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7" status: 404 len: 111 microversion: 2.1 time: 0.039809 Oct 30 01:07:16.555897 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 585/1194] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:07:16 2022] GET /compute/v2.1/servers/6f5485fd-b825-45e7-a5be-eb96208c09b7 => generated 111 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 30 01:07:16.562251 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-5ff486e7-df55-4908-b1de-b24d9106da99 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] Calling method '.version_select at 0x7f0eabaf7160>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:07:16.583421 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-5ff486e7-df55-4908-b1de-b24d9106da99 tempest-TestMinimumBasicScenario-1933647695 tempest-TestMinimumBasicScenario-1933647695-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-1028328605" status: 202 len: 0 microversion: 2.1 time: 0.025437 Oct 30 01:07:16.587330 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 610/1195] 10.222.0.16 () {58 vars in 1272 bytes} [Sun Oct 30 01:07:16 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-1028328605 => generated 0 bytes in 30 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 30 01:07:19.548241 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-5b0cabdb-5a07-455c-b49e-e64bc7e895ae tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:07:19.551201 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-5b0cabdb-5a07-455c-b49e-e64bc7e895ae tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: d95ac907-83b9-4fe2-b49a-defbe7240d3a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:07:19.558324 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5b0cabdb-5a07-455c-b49e-e64bc7e895ae tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:19.559466 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5b0cabdb-5a07-455c-b49e-e64bc7e895ae tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:19.560251 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5b0cabdb-5a07-455c-b49e-e64bc7e895ae tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:19.561178 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5b0cabdb-5a07-455c-b49e-e64bc7e895ae tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:19.561841 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5b0cabdb-5a07-455c-b49e-e64bc7e895ae tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:19.562451 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5b0cabdb-5a07-455c-b49e-e64bc7e895ae tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:19.602590 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-5b0cabdb-5a07-455c-b49e-e64bc7e895ae tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058529 Oct 30 01:07:19.610271 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 586/1196] 10.222.0.16 () {58 vars in 1302 bytes} [Sun Oct 30 01:07:19 2022] GET /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a/os-volume_attachments => generated 197 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:07:19.615401 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-2f87819c-a30b-492b-9f7b-f2e3ec1d825e tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:07:19.616008 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-2f87819c-a30b-492b-9f7b-f2e3ec1d825e tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: d95ac907-83b9-4fe2-b49a-defbe7240d3a] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:07:19.623502 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2f87819c-a30b-492b-9f7b-f2e3ec1d825e tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:19.624241 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2f87819c-a30b-492b-9f7b-f2e3ec1d825e tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:19.624887 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2f87819c-a30b-492b-9f7b-f2e3ec1d825e tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:19.625813 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2f87819c-a30b-492b-9f7b-f2e3ec1d825e tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:19.626454 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2f87819c-a30b-492b-9f7b-f2e3ec1d825e tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:19.627318 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2f87819c-a30b-492b-9f7b-f2e3ec1d825e tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:19.678060 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-2f87819c-a30b-492b-9f7b-f2e3ec1d825e tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a/os-volume_attachments/90c927cc-c3d0-46e0-b10f-c90b6a95232a" status: 200 len: 194 microversion: 2.1 time: 0.066576 Oct 30 01:07:19.680082 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 611/1197] 10.222.0.16 () {58 vars in 1413 bytes} [Sun Oct 30 01:07:19 2022] GET /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a/os-volume_attachments/90c927cc-c3d0-46e0-b10f-c90b6a95232a => generated 194 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:07:20.035402 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-53d3d4a3-3ffa-4165-813a-5d58a8f54d5b tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Action: 'create', calling method: .version_select at 0x7f0eb077db80>, body: {"keypair": {"name": "tempest-TestVolumeBootPattern-1884997751", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKuSJi9jU03ofqGMcmjn1XxPRSH0XHeBVYPOPHUHbXTrNoYUhNxAvTqoXE+cQeA6DgAoQqNJWUbM0jXmjpvVPxvuvAxe0ubGEvrrxcIY27HBBv0Ohs4hk/ZsJ3q1vkB7eA=="}} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:07:20.074558 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.quota [None req-53d3d4a3-3ffa-4165-813a-5d58a8f54d5b tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Getting quotas for project 238d71be4e094cb6b84f9a48a8a2c79c. Resources: {'key_pairs'} {{(pid=118962) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 30 01:07:20.083397 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.quota [None req-53d3d4a3-3ffa-4165-813a-5d58a8f54d5b tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Getting quotas for user 96e0cd00e4ef4a2da960aa5d4f66972a and project 238d71be4e094cb6b84f9a48a8a2c79c. Resources: {'key_pairs'} {{(pid=118962) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 30 01:07:20.147878 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-53d3d4a3-3ffa-4165-813a-5d58a8f54d5b tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 400 microversion: 2.1 time: 0.291543 Oct 30 01:07:20.149149 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 587/1198] 10.222.0.16 () {60 vars in 1159 bytes} [Sun Oct 30 01:07:19 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:07:21.331117 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-bee436a4-81db-4fd8-a357-d558c408d334 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "bd3202ae-e413-42f8-8ac8-0bc6dcdf63e0"}} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:07:21.335597 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-bee436a4-81db-4fd8-a357-d558c408d334 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: d95ac907-83b9-4fe2-b49a-defbe7240d3a] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:07:21.370653 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bee436a4-81db-4fd8-a357-d558c408d334 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:21.371621 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bee436a4-81db-4fd8-a357-d558c408d334 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.025s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:21.377865 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bee436a4-81db-4fd8-a357-d558c408d334 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:21.378761 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bee436a4-81db-4fd8-a357-d558c408d334 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:21.379255 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bee436a4-81db-4fd8-a357-d558c408d334 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:21.379695 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bee436a4-81db-4fd8-a357-d558c408d334 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:21.654960 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell1:SELECT=99,nova_cell1:UPDATE=1 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:07:21.880742 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-bee436a4-81db-4fd8-a357-d558c408d334 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "POST /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.564299 Oct 30 01:07:21.881839 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 612/1199] 10.222.0.16 () {60 vars in 1323 bytes} [Sun Oct 30 01:07:21 2022] POST /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a/os-volume_attachments => generated 194 bytes in 567 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:07:29.542845 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-4004b807-2c89-4c0d-8234-c708c3f23d0e tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:07:29.545366 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-4004b807-2c89-4c0d-8234-c708c3f23d0e tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: d95ac907-83b9-4fe2-b49a-defbe7240d3a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:07:29.552126 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4004b807-2c89-4c0d-8234-c708c3f23d0e tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:29.552627 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4004b807-2c89-4c0d-8234-c708c3f23d0e tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:29.553057 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4004b807-2c89-4c0d-8234-c708c3f23d0e tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:29.553731 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4004b807-2c89-4c0d-8234-c708c3f23d0e tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:29.554215 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4004b807-2c89-4c0d-8234-c708c3f23d0e tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:29.554628 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4004b807-2c89-4c0d-8234-c708c3f23d0e tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:29.589792 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-4004b807-2c89-4c0d-8234-c708c3f23d0e tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a/os-volume_attachments" status: 200 len: 371 microversion: 2.1 time: 0.051814 Oct 30 01:07:29.590460 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 588/1200] 10.222.0.16 () {58 vars in 1302 bytes} [Sun Oct 30 01:07:29 2022] GET /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a/os-volume_attachments => generated 371 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:07:29.744505 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-79d3feb9-b390-4525-a5a9-3fe7b82221cb tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:07:29.745045 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-79d3feb9-b390-4525-a5a9-3fe7b82221cb tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: d95ac907-83b9-4fe2-b49a-defbe7240d3a] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:07:29.751828 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-79d3feb9-b390-4525-a5a9-3fe7b82221cb tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:29.752317 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-79d3feb9-b390-4525-a5a9-3fe7b82221cb tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:29.752787 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-79d3feb9-b390-4525-a5a9-3fe7b82221cb tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:29.753625 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-79d3feb9-b390-4525-a5a9-3fe7b82221cb tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:29.754077 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-79d3feb9-b390-4525-a5a9-3fe7b82221cb tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:29.754495 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-79d3feb9-b390-4525-a5a9-3fe7b82221cb tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:29.784819 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-79d3feb9-b390-4525-a5a9-3fe7b82221cb tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a/os-volume_attachments/90c927cc-c3d0-46e0-b10f-c90b6a95232a" status: 200 len: 194 microversion: 2.1 time: 0.045515 Oct 30 01:07:29.785418 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 613/1201] 10.222.0.16 () {58 vars in 1413 bytes} [Sun Oct 30 01:07:29 2022] GET /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a/os-volume_attachments/90c927cc-c3d0-46e0-b10f-c90b6a95232a => generated 194 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:07:29.800782 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-7c840eee-c72e-4a6e-b186-d67045bf869a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:07:29.801272 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-7c840eee-c72e-4a6e-b186-d67045bf869a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: d95ac907-83b9-4fe2-b49a-defbe7240d3a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:07:29.808135 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7c840eee-c72e-4a6e-b186-d67045bf869a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:29.808621 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7c840eee-c72e-4a6e-b186-d67045bf869a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:29.809052 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7c840eee-c72e-4a6e-b186-d67045bf869a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:29.809751 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7c840eee-c72e-4a6e-b186-d67045bf869a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:29.810196 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7c840eee-c72e-4a6e-b186-d67045bf869a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:29.810611 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7c840eee-c72e-4a6e-b186-d67045bf869a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:29.830535 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:07:29.899303 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.objects.instance [None req-7c840eee-c72e-4a6e-b186-d67045bf869a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lazy-loading 'services' on Instance uuid d95ac907-83b9-4fe2-b49a-defbe7240d3a {{(pid=118962) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 30 01:07:29.977783 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-7c840eee-c72e-4a6e-b186-d67045bf869a tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "DELETE /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a/os-volume_attachments/90c927cc-c3d0-46e0-b10f-c90b6a95232a" status: 202 len: 0 microversion: 2.1 time: 0.183175 Oct 30 01:07:29.978406 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 589/1202] 10.222.0.16 () {58 vars in 1416 bytes} [Sun Oct 30 01:07:29 2022] DELETE /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a/os-volume_attachments/90c927cc-c3d0-46e0-b10f-c90b6a95232a => generated 0 bytes in 185 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 30 01:07:32.189527 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-c65edf6a-c826-4b39-adcc-8cc9606790f9 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:07:32.190047 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-c65edf6a-c826-4b39-adcc-8cc9606790f9 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: d95ac907-83b9-4fe2-b49a-defbe7240d3a] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:07:32.196939 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c65edf6a-c826-4b39-adcc-8cc9606790f9 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:32.197422 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c65edf6a-c826-4b39-adcc-8cc9606790f9 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:32.197852 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c65edf6a-c826-4b39-adcc-8cc9606790f9 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:32.198685 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c65edf6a-c826-4b39-adcc-8cc9606790f9 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:32.198999 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c65edf6a-c826-4b39-adcc-8cc9606790f9 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:32.199418 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c65edf6a-c826-4b39-adcc-8cc9606790f9 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:32.231034 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-c65edf6a-c826-4b39-adcc-8cc9606790f9 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a/os-volume_attachments/bd3202ae-e413-42f8-8ac8-0bc6dcdf63e0" status: 200 len: 194 microversion: 2.1 time: 0.045469 Oct 30 01:07:32.231645 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 614/1203] 10.222.0.16 () {58 vars in 1413 bytes} [Sun Oct 30 01:07:32 2022] GET /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a/os-volume_attachments/bd3202ae-e413-42f8-8ac8-0bc6dcdf63e0 => generated 194 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:07:32.239752 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell0:SELECT=4 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:07:32.245149 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-0ea6b093-9896-47b8-a215-9b5b503b275c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:07:32.245797 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-0ea6b093-9896-47b8-a215-9b5b503b275c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: d95ac907-83b9-4fe2-b49a-defbe7240d3a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:07:32.247829 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_api:SELECT=49,nova_api:UPDATE=1,nova_api:INSERT=1 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:07:32.253693 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0ea6b093-9896-47b8-a215-9b5b503b275c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:32.254177 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0ea6b093-9896-47b8-a215-9b5b503b275c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:32.254612 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0ea6b093-9896-47b8-a215-9b5b503b275c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:32.255315 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0ea6b093-9896-47b8-a215-9b5b503b275c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:32.255749 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0ea6b093-9896-47b8-a215-9b5b503b275c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:32.256186 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0ea6b093-9896-47b8-a215-9b5b503b275c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:32.345276 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.objects.instance [None req-0ea6b093-9896-47b8-a215-9b5b503b275c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lazy-loading 'services' on Instance uuid d95ac907-83b9-4fe2-b49a-defbe7240d3a {{(pid=118962) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 30 01:07:32.430843 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-0ea6b093-9896-47b8-a215-9b5b503b275c tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "DELETE /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a/os-volume_attachments/bd3202ae-e413-42f8-8ac8-0bc6dcdf63e0" status: 202 len: 0 microversion: 2.1 time: 0.191784 Oct 30 01:07:32.431679 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 590/1204] 10.222.0.16 () {58 vars in 1416 bytes} [Sun Oct 30 01:07:32 2022] DELETE /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a/os-volume_attachments/bd3202ae-e413-42f8-8ac8-0bc6dcdf63e0 => generated 0 bytes in 194 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 30 01:07:33.617578 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-efcd3e10-accc-42b1-bc4f-0acfc32f9aed tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:07:33.620086 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-efcd3e10-accc-42b1-bc4f-0acfc32f9aed tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: d95ac907-83b9-4fe2-b49a-defbe7240d3a] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:07:33.626555 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-efcd3e10-accc-42b1-bc4f-0acfc32f9aed tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:33.627038 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-efcd3e10-accc-42b1-bc4f-0acfc32f9aed tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:33.627471 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-efcd3e10-accc-42b1-bc4f-0acfc32f9aed tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:33.628216 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-efcd3e10-accc-42b1-bc4f-0acfc32f9aed tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:33.628666 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-efcd3e10-accc-42b1-bc4f-0acfc32f9aed tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:33.629086 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-efcd3e10-accc-42b1-bc4f-0acfc32f9aed tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:33.657508 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-efcd3e10-accc-42b1-bc4f-0acfc32f9aed tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.043897 Oct 30 01:07:33.658071 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 615/1205] 10.222.0.16 () {58 vars in 1302 bytes} [Sun Oct 30 01:07:33 2022] GET /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a/os-volume_attachments => generated 25 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 30 01:07:33.785844 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-729fa1f2-d3f3-4e92-80b4-e270364fd537 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:07:33.789292 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-729fa1f2-d3f3-4e92-80b4-e270364fd537 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: d95ac907-83b9-4fe2-b49a-defbe7240d3a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:07:33.796497 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-729fa1f2-d3f3-4e92-80b4-e270364fd537 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:33.796971 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-729fa1f2-d3f3-4e92-80b4-e270364fd537 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:33.797392 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-729fa1f2-d3f3-4e92-80b4-e270364fd537 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:33.798109 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-729fa1f2-d3f3-4e92-80b4-e270364fd537 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:33.798547 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-729fa1f2-d3f3-4e92-80b4-e270364fd537 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:33.798958 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-729fa1f2-d3f3-4e92-80b4-e270364fd537 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:33.829449 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-729fa1f2-d3f3-4e92-80b4-e270364fd537 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.051549 Oct 30 01:07:33.829940 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 591/1206] 10.222.0.16 () {58 vars in 1302 bytes} [Sun Oct 30 01:07:33 2022] GET /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a/os-volume_attachments => generated 25 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 30 01:07:33.896444 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-2f044a34-c57d-416c-87a5-92d82ed16ca7 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:07:33.896947 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-2f044a34-c57d-416c-87a5-92d82ed16ca7 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: d95ac907-83b9-4fe2-b49a-defbe7240d3a] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:07:33.903394 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2f044a34-c57d-416c-87a5-92d82ed16ca7 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:33.903919 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2f044a34-c57d-416c-87a5-92d82ed16ca7 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:33.904360 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2f044a34-c57d-416c-87a5-92d82ed16ca7 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:33.905062 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2f044a34-c57d-416c-87a5-92d82ed16ca7 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:33.905511 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2f044a34-c57d-416c-87a5-92d82ed16ca7 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:33.905945 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2f044a34-c57d-416c-87a5-92d82ed16ca7 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:33.932424 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-2f044a34-c57d-416c-87a5-92d82ed16ca7 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: d95ac907-83b9-4fe2-b49a-defbe7240d3a] Going to try to terminate instance {{(pid=118961) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 30 01:07:34.027233 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-2f044a34-c57d-416c-87a5-92d82ed16ca7 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "DELETE /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a" status: 204 len: 0 microversion: 2.1 time: 0.134427 Oct 30 01:07:34.027833 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 616/1207] 10.222.0.16 () {58 vars in 1239 bytes} [Sun Oct 30 01:07:33 2022] DELETE /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a => generated 0 bytes in 136 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 30 01:07:34.036162 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-ccc82c8e-aa4d-4a4d-9d68-5b33faf0d768 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:07:34.036780 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-ccc82c8e-aa4d-4a4d-9d68-5b33faf0d768 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: d95ac907-83b9-4fe2-b49a-defbe7240d3a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:07:34.043117 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ccc82c8e-aa4d-4a4d-9d68-5b33faf0d768 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:34.043601 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ccc82c8e-aa4d-4a4d-9d68-5b33faf0d768 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:34.044095 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ccc82c8e-aa4d-4a4d-9d68-5b33faf0d768 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:34.044789 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ccc82c8e-aa4d-4a4d-9d68-5b33faf0d768 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:34.045239 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ccc82c8e-aa4d-4a4d-9d68-5b33faf0d768 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:34.045674 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ccc82c8e-aa4d-4a4d-9d68-5b33faf0d768 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:34.191908 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-ccc82c8e-aa4d-4a4d-9d68-5b33faf0d768 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:07:34.201046 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-ccc82c8e-aa4d-4a4d-9d68-5b33faf0d768 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a" status: 200 len: 1674 microversion: 2.1 time: 0.168752 Oct 30 01:07:34.201625 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 592/1208] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:07:34 2022] GET /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a => generated 1674 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:07:35.225786 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-39379f8c-cf0a-4532-a62a-e9d7a2051d3b tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:07:35.226488 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-39379f8c-cf0a-4532-a62a-e9d7a2051d3b tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: d95ac907-83b9-4fe2-b49a-defbe7240d3a] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:07:35.233844 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-39379f8c-cf0a-4532-a62a-e9d7a2051d3b tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:35.234353 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-39379f8c-cf0a-4532-a62a-e9d7a2051d3b tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:35.234814 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-39379f8c-cf0a-4532-a62a-e9d7a2051d3b tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:35.235542 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-39379f8c-cf0a-4532-a62a-e9d7a2051d3b tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:35.236048 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-39379f8c-cf0a-4532-a62a-e9d7a2051d3b tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:35.236490 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-39379f8c-cf0a-4532-a62a-e9d7a2051d3b tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:35.386476 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-39379f8c-cf0a-4532-a62a-e9d7a2051d3b tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:07:35.395214 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-39379f8c-cf0a-4532-a62a-e9d7a2051d3b tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a" status: 200 len: 1674 microversion: 2.1 time: 0.174044 Oct 30 01:07:35.395946 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 617/1209] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:07:35 2022] GET /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a => generated 1674 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:07:36.419289 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-e2d6ef69-f50a-4bcb-b0ff-a5e5030ee203 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:07:36.419857 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-e2d6ef69-f50a-4bcb-b0ff-a5e5030ee203 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: d95ac907-83b9-4fe2-b49a-defbe7240d3a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:07:36.426762 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e2d6ef69-f50a-4bcb-b0ff-a5e5030ee203 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:36.427244 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e2d6ef69-f50a-4bcb-b0ff-a5e5030ee203 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:36.427677 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e2d6ef69-f50a-4bcb-b0ff-a5e5030ee203 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:36.428396 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e2d6ef69-f50a-4bcb-b0ff-a5e5030ee203 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:36.428843 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e2d6ef69-f50a-4bcb-b0ff-a5e5030ee203 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:36.429264 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e2d6ef69-f50a-4bcb-b0ff-a5e5030ee203 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:36.564620 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-e2d6ef69-f50a-4bcb-b0ff-a5e5030ee203 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:07:36.572362 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-e2d6ef69-f50a-4bcb-b0ff-a5e5030ee203 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a" status: 200 len: 1674 microversion: 2.1 time: 0.157232 Oct 30 01:07:36.572927 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 593/1210] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:07:36 2022] GET /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a => generated 1674 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:07:37.494356 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [req-37e6bf32-47b6-4c62-b0c5-8c7c6e2951f3 req-c1d9ac0f-a68f-4a5b-bf56-7e331358d51b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d95ac907-83b9-4fe2-b49a-defbe7240d3a", "name": "network-vif-deleted", "tag": "b0c592fa-975d-4695-84db-c884d520eff2"}]} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:07:37.500645 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_api:SELECT=44,nova_api:DELETE=1,nova_api:UPDATE=1 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:07:37.506441 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-37e6bf32-47b6-4c62-b0c5-8c7c6e2951f3 req-c1d9ac0f-a68f-4a5b-bf56-7e331358d51b service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:37.506925 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-37e6bf32-47b6-4c62-b0c5-8c7c6e2951f3 req-c1d9ac0f-a68f-4a5b-bf56-7e331358d51b service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:37.507358 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-37e6bf32-47b6-4c62-b0c5-8c7c6e2951f3 req-c1d9ac0f-a68f-4a5b-bf56-7e331358d51b service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:37.520556 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.compute.server_external_events [req-37e6bf32-47b6-4c62-b0c5-8c7c6e2951f3 req-c1d9ac0f-a68f-4a5b-bf56-7e331358d51b service nova] Creating event network-vif-deleted:b0c592fa-975d-4695-84db-c884d520eff2 for instance d95ac907-83b9-4fe2-b49a-defbe7240d3a on hv-cinder-maste Oct 30 01:07:37.527359 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [req-37e6bf32-47b6-4c62-b0c5-8c7c6e2951f3 req-c1d9ac0f-a68f-4a5b-bf56-7e331358d51b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036785 Oct 30 01:07:37.528014 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 618/1211] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:07:37 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:07:37.597065 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-139d4b2d-e1d8-4295-ac4a-b3b672f3247e tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:07:37.597678 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-139d4b2d-e1d8-4295-ac4a-b3b672f3247e tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: d95ac907-83b9-4fe2-b49a-defbe7240d3a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:07:37.607299 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-139d4b2d-e1d8-4295-ac4a-b3b672f3247e tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:37.608088 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-139d4b2d-e1d8-4295-ac4a-b3b672f3247e tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:37.608844 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-139d4b2d-e1d8-4295-ac4a-b3b672f3247e tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:37.610239 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-139d4b2d-e1d8-4295-ac4a-b3b672f3247e tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:37.610840 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-139d4b2d-e1d8-4295-ac4a-b3b672f3247e tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:37.611600 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-139d4b2d-e1d8-4295-ac4a-b3b672f3247e tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:37.714628 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-139d4b2d-e1d8-4295-ac4a-b3b672f3247e tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0e5bd11ac14259ad376ce54bcb7025', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bf407954ed34ebfaf56db206df4fc84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:07:37.725721 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-139d4b2d-e1d8-4295-ac4a-b3b672f3247e tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a" status: 200 len: 1323 microversion: 2.1 time: 0.132177 Oct 30 01:07:37.740429 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 594/1212] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:07:37 2022] GET /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a => generated 1323 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:07:38.751354 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-a7958d5d-450c-4d82-bea7-5d5b58eef94f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:07:38.752069 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-a7958d5d-450c-4d82-bea7-5d5b58eef94f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: d95ac907-83b9-4fe2-b49a-defbe7240d3a] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:07:38.758875 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a7958d5d-450c-4d82-bea7-5d5b58eef94f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:38.759459 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a7958d5d-450c-4d82-bea7-5d5b58eef94f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:38.759921 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a7958d5d-450c-4d82-bea7-5d5b58eef94f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:38.760618 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a7958d5d-450c-4d82-bea7-5d5b58eef94f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:38.761069 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a7958d5d-450c-4d82-bea7-5d5b58eef94f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:38.761487 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a7958d5d-450c-4d82-bea7-5d5b58eef94f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:38.774906 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.wsgi [None req-a7958d5d-450c-4d82-bea7-5d5b58eef94f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] HTTP exception thrown: Instance d95ac907-83b9-4fe2-b49a-defbe7240d3a could not be found. Oct 30 01:07:38.776686 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-a7958d5d-450c-4d82-bea7-5d5b58eef94f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Request method failure captured: Oct 30 01:07:38.776686 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: request: GET /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a HTTP/1.1 Oct 30 01:07:38.776686 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Accept: application/json Oct 30 01:07:38.776686 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Accept-Encoding: identity Oct 30 01:07:38.776686 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Connection: close Oct 30 01:07:38.776686 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Content-Length: 0 Oct 30 01:07:38.776686 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Content-Type: application/json Oct 30 01:07:38.776686 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Host: 10.222.0.16 Oct 30 01:07:38.776686 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Openstack-System-Scope: None Oct 30 01:07:38.776686 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: User-Agent: python-urllib3/1.26.12 Oct 30 01:07:38.776686 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Auth-Token: gAAAAABjXcufJywtdFM5PObVwvc4XfvnADFZy4zohCugYTCT6kU4WYIm9tEeU-k9LqQ38owLu40IDCuTtQtYT30Plcr8MKntgwthetV3Q2djsPe7ErJwj43_xWiRbN29t1NWovB4v2ZaEkG-yf_zjlv7AkC5WmwDZXliUm7DodcvaVfundKFDhc Oct 30 01:07:38.776686 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Domain-Id: None Oct 30 01:07:38.776686 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Domain-Name: None Oct 30 01:07:38.776686 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Identity-Status: Confirmed Oct 30 01:07:38.776686 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Is-Admin-Project: True Oct 30 01:07:38.776686 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Domain-Id: default Oct 30 01:07:38.776686 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Domain-Name: Default Oct 30 01:07:38.776686 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Id: 8bf407954ed34ebfaf56db206df4fc84 Oct 30 01:07:38.776686 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Name: tempest-AttachVolumeTestJSON-298540627 Oct 30 01:07:38.776686 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Role: reader,member Oct 30 01:07:38.776686 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Roles: reader,member Oct 30 01:07:38.778374 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_8bf407954ed34ebfaf56db206df4fc84"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8bf407954ed34ebfaf56db206df4fc84"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/8bf407954ed34ebfaf56db206df4fc84"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8bf407954ed34ebfaf56db206df4fc84"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 01:07:38.778374 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant: tempest-AttachVolumeTestJSON-298540627 Oct 30 01:07:38.778374 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant-Id: 8bf407954ed34ebfaf56db206df4fc84 Oct 30 01:07:38.778374 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant-Name: tempest-AttachVolumeTestJSON-298540627 Oct 30 01:07:38.778374 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User: tempest-AttachVolumeTestJSON-298540627-project Oct 30 01:07:38.778374 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Domain-Id: default Oct 30 01:07:38.778374 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Domain-Name: Default Oct 30 01:07:38.778374 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Id: 1c0e5bd11ac14259ad376ce54bcb7025 Oct 30 01:07:38.778374 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Name: tempest-AttachVolumeTestJSON-298540627-project Oct 30 01:07:38.778374 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: method: > Oct 30 01:07:38.778374 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: exception: Instance d95ac907-83b9-4fe2-b49a-defbe7240d3a could not be found. Oct 30 01:07:38.778374 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: action_args: {'id': 'd95ac907-83b9-4fe2-b49a-defbe7240d3a'} Oct 30 01:07:38.779325 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 01:07:38.779325 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:07:38.779325 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 30 01:07:38.779325 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 30 01:07:38.779325 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 30 01:07:38.779325 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 30 01:07:38.779325 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 30 01:07:38.779325 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 30 01:07:38.779325 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 30 01:07:38.779325 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 30 01:07:38.779325 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d95ac907-83b9-4fe2-b49a-defbe7240d3a could not be found. Oct 30 01:07:38.779325 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:07:38.779325 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:07:38.779325 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:07:38.779325 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:07:38.779325 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:07:38.779325 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:07:38.779325 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 01:07:38.779325 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 01:07:38.779325 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 01:07:38.780902 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 01:07:38.780902 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 30 01:07:38.780902 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 30 01:07:38.780902 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 30 01:07:38.780902 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 30 01:07:38.780902 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 30 01:07:38.780902 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 30 01:07:38.780902 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d95ac907-83b9-4fe2-b49a-defbe7240d3a could not be found. Oct 30 01:07:38.780902 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:07:38.780902 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:07:38.780902 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:07:38.780902 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:07:38.780902 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:07:38.780902 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:07:38.780902 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 01:07:38.780902 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 01:07:38.780902 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d95ac907-83b9-4fe2-b49a-defbe7240d3a could not be found. Oct 30 01:07:38.780902 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:07:38.782265 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-a7958d5d-450c-4d82-bea7-5d5b58eef94f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Returning 404 to user: Instance d95ac907-83b9-4fe2-b49a-defbe7240d3a could not be found. {{(pid=118961) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 01:07:38.782265 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-a7958d5d-450c-4d82-bea7-5d5b58eef94f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a" status: 404 len: 111 microversion: 2.1 time: 0.032409 Oct 30 01:07:38.782265 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 619/1213] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:07:38 2022] GET /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 30 01:07:38.787113 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-6e613db8-56db-4277-9091-7ef7ae163a22 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '.version_select at 0x7f0eafbf03a0>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:07:38.810144 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-6e613db8-56db-4277-9091-7ef7ae163a22 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1936244065" status: 202 len: 0 microversion: 2.1 time: 0.026880 Oct 30 01:07:38.810788 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 595/1214] 10.222.0.16 () {58 vars in 1221 bytes} [Sun Oct 30 01:07:38 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1936244065 => generated 0 bytes in 28 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 30 01:07:41.751233 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-e25f896e-665c-47e4-9790-88099a230a44 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:07:41.751757 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-e25f896e-665c-47e4-9790-88099a230a44 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: d95ac907-83b9-4fe2-b49a-defbe7240d3a] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:07:41.758632 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e25f896e-665c-47e4-9790-88099a230a44 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:41.759117 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e25f896e-665c-47e4-9790-88099a230a44 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:41.759552 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e25f896e-665c-47e4-9790-88099a230a44 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:41.760290 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e25f896e-665c-47e4-9790-88099a230a44 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:41.760743 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e25f896e-665c-47e4-9790-88099a230a44 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:41.761161 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e25f896e-665c-47e4-9790-88099a230a44 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:41.775499 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.wsgi [None req-e25f896e-665c-47e4-9790-88099a230a44 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] HTTP exception thrown: Instance d95ac907-83b9-4fe2-b49a-defbe7240d3a could not be found. Oct 30 01:07:41.777086 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-e25f896e-665c-47e4-9790-88099a230a44 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Request method failure captured: Oct 30 01:07:41.777086 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: request: DELETE /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a HTTP/1.1 Oct 30 01:07:41.777086 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Accept: application/json Oct 30 01:07:41.777086 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Accept-Encoding: identity Oct 30 01:07:41.777086 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Connection: close Oct 30 01:07:41.777086 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Content-Length: 0 Oct 30 01:07:41.777086 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Content-Type: application/json Oct 30 01:07:41.777086 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Host: 10.222.0.16 Oct 30 01:07:41.777086 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Openstack-System-Scope: None Oct 30 01:07:41.777086 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: User-Agent: python-urllib3/1.26.12 Oct 30 01:07:41.777086 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Auth-Token: gAAAAABjXcufJywtdFM5PObVwvc4XfvnADFZy4zohCugYTCT6kU4WYIm9tEeU-k9LqQ38owLu40IDCuTtQtYT30Plcr8MKntgwthetV3Q2djsPe7ErJwj43_xWiRbN29t1NWovB4v2ZaEkG-yf_zjlv7AkC5WmwDZXliUm7DodcvaVfundKFDhc Oct 30 01:07:41.777086 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Domain-Id: None Oct 30 01:07:41.777086 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Domain-Name: None Oct 30 01:07:41.777086 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Identity-Status: Confirmed Oct 30 01:07:41.777086 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Is-Admin-Project: True Oct 30 01:07:41.777086 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Domain-Id: default Oct 30 01:07:41.777086 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Domain-Name: Default Oct 30 01:07:41.777086 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Id: 8bf407954ed34ebfaf56db206df4fc84 Oct 30 01:07:41.777086 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Name: tempest-AttachVolumeTestJSON-298540627 Oct 30 01:07:41.777086 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Role: reader,member Oct 30 01:07:41.777086 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Roles: reader,member Oct 30 01:07:41.778659 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_8bf407954ed34ebfaf56db206df4fc84"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8bf407954ed34ebfaf56db206df4fc84"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/8bf407954ed34ebfaf56db206df4fc84"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8bf407954ed34ebfaf56db206df4fc84"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 01:07:41.778659 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant: tempest-AttachVolumeTestJSON-298540627 Oct 30 01:07:41.778659 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant-Id: 8bf407954ed34ebfaf56db206df4fc84 Oct 30 01:07:41.778659 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant-Name: tempest-AttachVolumeTestJSON-298540627 Oct 30 01:07:41.778659 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User: tempest-AttachVolumeTestJSON-298540627-project Oct 30 01:07:41.778659 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Domain-Id: default Oct 30 01:07:41.778659 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Domain-Name: Default Oct 30 01:07:41.778659 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Id: 1c0e5bd11ac14259ad376ce54bcb7025 Oct 30 01:07:41.778659 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Name: tempest-AttachVolumeTestJSON-298540627-project Oct 30 01:07:41.778659 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: method: > Oct 30 01:07:41.778659 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: exception: Instance d95ac907-83b9-4fe2-b49a-defbe7240d3a could not be found. Oct 30 01:07:41.778659 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: action_args: {'id': 'd95ac907-83b9-4fe2-b49a-defbe7240d3a'} Oct 30 01:07:41.779551 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 01:07:41.779551 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:07:41.779551 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 30 01:07:41.779551 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 30 01:07:41.779551 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 30 01:07:41.779551 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 30 01:07:41.779551 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 30 01:07:41.779551 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 30 01:07:41.779551 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 30 01:07:41.779551 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 30 01:07:41.779551 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d95ac907-83b9-4fe2-b49a-defbe7240d3a could not be found. Oct 30 01:07:41.779551 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:07:41.779551 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:07:41.779551 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:07:41.779551 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:07:41.779551 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:07:41.779551 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:07:41.779551 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 01:07:41.779551 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 01:07:41.779551 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 01:07:41.781160 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 01:07:41.781160 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 30 01:07:41.781160 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 30 01:07:41.781160 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 30 01:07:41.781160 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 30 01:07:41.781160 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 30 01:07:41.781160 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 30 01:07:41.781160 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 30 01:07:41.781160 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 30 01:07:41.781160 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d95ac907-83b9-4fe2-b49a-defbe7240d3a could not be found. Oct 30 01:07:41.781160 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:07:41.781160 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:07:41.781160 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:07:41.781160 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:07:41.781160 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:07:41.781160 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:07:41.781160 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 01:07:41.781160 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 01:07:41.781160 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d95ac907-83b9-4fe2-b49a-defbe7240d3a could not be found. Oct 30 01:07:41.781160 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:07:41.782567 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-e25f896e-665c-47e4-9790-88099a230a44 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Returning 404 to user: Instance d95ac907-83b9-4fe2-b49a-defbe7240d3a could not be found. {{(pid=118961) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 01:07:41.782567 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-e25f896e-665c-47e4-9790-88099a230a44 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "DELETE /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a" status: 404 len: 111 microversion: 2.1 time: 0.032556 Oct 30 01:07:41.782567 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 620/1215] 10.222.0.16 () {58 vars in 1239 bytes} [Sun Oct 30 01:07:41 2022] DELETE /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 30 01:07:41.788030 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-32f2d74d-54ec-4679-8467-72500d2670a7 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:07:41.788671 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-32f2d74d-54ec-4679-8467-72500d2670a7 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: d95ac907-83b9-4fe2-b49a-defbe7240d3a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:07:41.795400 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-32f2d74d-54ec-4679-8467-72500d2670a7 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:41.796147 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-32f2d74d-54ec-4679-8467-72500d2670a7 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:41.796587 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-32f2d74d-54ec-4679-8467-72500d2670a7 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:41.797273 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-32f2d74d-54ec-4679-8467-72500d2670a7 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:41.797720 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-32f2d74d-54ec-4679-8467-72500d2670a7 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:41.798137 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-32f2d74d-54ec-4679-8467-72500d2670a7 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:41.811459 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.wsgi [None req-32f2d74d-54ec-4679-8467-72500d2670a7 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] HTTP exception thrown: Instance d95ac907-83b9-4fe2-b49a-defbe7240d3a could not be found. Oct 30 01:07:41.813029 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-32f2d74d-54ec-4679-8467-72500d2670a7 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Request method failure captured: Oct 30 01:07:41.813029 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: request: GET /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a HTTP/1.1 Oct 30 01:07:41.813029 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept: application/json Oct 30 01:07:41.813029 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept-Encoding: identity Oct 30 01:07:41.813029 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Connection: close Oct 30 01:07:41.813029 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Length: 0 Oct 30 01:07:41.813029 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Type: application/json Oct 30 01:07:41.813029 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Host: 10.222.0.16 Oct 30 01:07:41.813029 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Openstack-System-Scope: None Oct 30 01:07:41.813029 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: User-Agent: python-urllib3/1.26.12 Oct 30 01:07:41.813029 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Auth-Token: gAAAAABjXcufJywtdFM5PObVwvc4XfvnADFZy4zohCugYTCT6kU4WYIm9tEeU-k9LqQ38owLu40IDCuTtQtYT30Plcr8MKntgwthetV3Q2djsPe7ErJwj43_xWiRbN29t1NWovB4v2ZaEkG-yf_zjlv7AkC5WmwDZXliUm7DodcvaVfundKFDhc Oct 30 01:07:41.813029 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Id: None Oct 30 01:07:41.813029 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Name: None Oct 30 01:07:41.813029 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Identity-Status: Confirmed Oct 30 01:07:41.813029 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Is-Admin-Project: True Oct 30 01:07:41.813029 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Id: default Oct 30 01:07:41.813029 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Name: Default Oct 30 01:07:41.813029 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Id: 8bf407954ed34ebfaf56db206df4fc84 Oct 30 01:07:41.813029 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Name: tempest-AttachVolumeTestJSON-298540627 Oct 30 01:07:41.813029 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Role: reader,member Oct 30 01:07:41.813029 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Roles: reader,member Oct 30 01:07:41.814613 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_8bf407954ed34ebfaf56db206df4fc84"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8bf407954ed34ebfaf56db206df4fc84"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/8bf407954ed34ebfaf56db206df4fc84"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8bf407954ed34ebfaf56db206df4fc84"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 01:07:41.814613 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant: tempest-AttachVolumeTestJSON-298540627 Oct 30 01:07:41.814613 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Id: 8bf407954ed34ebfaf56db206df4fc84 Oct 30 01:07:41.814613 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Name: tempest-AttachVolumeTestJSON-298540627 Oct 30 01:07:41.814613 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User: tempest-AttachVolumeTestJSON-298540627-project Oct 30 01:07:41.814613 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Id: default Oct 30 01:07:41.814613 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Name: Default Oct 30 01:07:41.814613 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Id: 1c0e5bd11ac14259ad376ce54bcb7025 Oct 30 01:07:41.814613 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Name: tempest-AttachVolumeTestJSON-298540627-project Oct 30 01:07:41.814613 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: method: > Oct 30 01:07:41.814613 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: exception: Instance d95ac907-83b9-4fe2-b49a-defbe7240d3a could not be found. Oct 30 01:07:41.814613 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: action_args: {'id': 'd95ac907-83b9-4fe2-b49a-defbe7240d3a'} Oct 30 01:07:41.815507 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 01:07:41.815507 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:07:41.815507 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 30 01:07:41.815507 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 30 01:07:41.815507 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 30 01:07:41.815507 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 30 01:07:41.815507 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 30 01:07:41.815507 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 30 01:07:41.815507 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 30 01:07:41.815507 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 30 01:07:41.815507 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d95ac907-83b9-4fe2-b49a-defbe7240d3a could not be found. Oct 30 01:07:41.815507 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:07:41.815507 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:07:41.815507 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:07:41.815507 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:07:41.815507 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:07:41.815507 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:07:41.815507 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 01:07:41.815507 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 01:07:41.815507 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 01:07:41.816993 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 01:07:41.816993 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 30 01:07:41.816993 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 30 01:07:41.816993 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 30 01:07:41.816993 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 30 01:07:41.816993 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 30 01:07:41.816993 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 30 01:07:41.816993 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d95ac907-83b9-4fe2-b49a-defbe7240d3a could not be found. Oct 30 01:07:41.816993 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:07:41.816993 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:07:41.816993 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:07:41.816993 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:07:41.816993 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:07:41.816993 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:07:41.816993 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 01:07:41.816993 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 01:07:41.816993 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d95ac907-83b9-4fe2-b49a-defbe7240d3a could not be found. Oct 30 01:07:41.816993 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:07:41.818578 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-32f2d74d-54ec-4679-8467-72500d2670a7 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Returning 404 to user: Instance d95ac907-83b9-4fe2-b49a-defbe7240d3a could not be found. {{(pid=118962) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 01:07:41.818578 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-32f2d74d-54ec-4679-8467-72500d2670a7 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a" status: 404 len: 111 microversion: 2.1 time: 0.030934 Oct 30 01:07:41.818578 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 596/1216] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:07:41 2022] GET /compute/v2.1/servers/d95ac907-83b9-4fe2-b49a-defbe7240d3a => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 30 01:07:42.970937 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-80ee9beb-0f82-4aba-a876-d7a5fdc8307f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:07:42.971427 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-80ee9beb-0f82-4aba-a876-d7a5fdc8307f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:07:42.978252 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-80ee9beb-0f82-4aba-a876-d7a5fdc8307f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:42.978737 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-80ee9beb-0f82-4aba-a876-d7a5fdc8307f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:42.979163 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-80ee9beb-0f82-4aba-a876-d7a5fdc8307f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:42.979944 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-80ee9beb-0f82-4aba-a876-d7a5fdc8307f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:42.980395 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-80ee9beb-0f82-4aba-a876-d7a5fdc8307f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:42.980819 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-80ee9beb-0f82-4aba-a876-d7a5fdc8307f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:42.995101 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.wsgi [None req-80ee9beb-0f82-4aba-a876-d7a5fdc8307f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] HTTP exception thrown: Instance 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 could not be found. Oct 30 01:07:42.996684 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-80ee9beb-0f82-4aba-a876-d7a5fdc8307f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Request method failure captured: Oct 30 01:07:42.996684 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: request: DELETE /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 HTTP/1.1 Oct 30 01:07:42.996684 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Accept: application/json Oct 30 01:07:42.996684 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Accept-Encoding: identity Oct 30 01:07:42.996684 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Connection: close Oct 30 01:07:42.996684 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Content-Length: 0 Oct 30 01:07:42.996684 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Content-Type: application/json Oct 30 01:07:42.996684 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Host: 10.222.0.16 Oct 30 01:07:42.996684 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Openstack-System-Scope: None Oct 30 01:07:42.996684 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: User-Agent: python-urllib3/1.26.12 Oct 30 01:07:42.996684 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Auth-Token: gAAAAABjXcufJywtdFM5PObVwvc4XfvnADFZy4zohCugYTCT6kU4WYIm9tEeU-k9LqQ38owLu40IDCuTtQtYT30Plcr8MKntgwthetV3Q2djsPe7ErJwj43_xWiRbN29t1NWovB4v2ZaEkG-yf_zjlv7AkC5WmwDZXliUm7DodcvaVfundKFDhc Oct 30 01:07:42.996684 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Domain-Id: None Oct 30 01:07:42.996684 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Domain-Name: None Oct 30 01:07:42.996684 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Identity-Status: Confirmed Oct 30 01:07:42.996684 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Is-Admin-Project: True Oct 30 01:07:42.996684 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Domain-Id: default Oct 30 01:07:42.996684 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Domain-Name: Default Oct 30 01:07:42.996684 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Id: 8bf407954ed34ebfaf56db206df4fc84 Oct 30 01:07:42.996684 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Name: tempest-AttachVolumeTestJSON-298540627 Oct 30 01:07:42.996684 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Role: reader,member Oct 30 01:07:42.996684 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Roles: reader,member Oct 30 01:07:42.998270 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_8bf407954ed34ebfaf56db206df4fc84"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8bf407954ed34ebfaf56db206df4fc84"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/8bf407954ed34ebfaf56db206df4fc84"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8bf407954ed34ebfaf56db206df4fc84"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 01:07:42.998270 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant: tempest-AttachVolumeTestJSON-298540627 Oct 30 01:07:42.998270 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant-Id: 8bf407954ed34ebfaf56db206df4fc84 Oct 30 01:07:42.998270 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant-Name: tempest-AttachVolumeTestJSON-298540627 Oct 30 01:07:42.998270 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User: tempest-AttachVolumeTestJSON-298540627-project Oct 30 01:07:42.998270 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Domain-Id: default Oct 30 01:07:42.998270 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Domain-Name: Default Oct 30 01:07:42.998270 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Id: 1c0e5bd11ac14259ad376ce54bcb7025 Oct 30 01:07:42.998270 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Name: tempest-AttachVolumeTestJSON-298540627-project Oct 30 01:07:42.998270 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: method: > Oct 30 01:07:42.998270 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: exception: Instance 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 could not be found. Oct 30 01:07:42.998270 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: action_args: {'id': '70ad3ec1-64bc-40ac-bb98-cc82d19e2a14'} Oct 30 01:07:42.999761 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 01:07:42.999761 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:07:42.999761 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 30 01:07:42.999761 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 30 01:07:42.999761 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 30 01:07:42.999761 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 30 01:07:42.999761 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 30 01:07:42.999761 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 30 01:07:42.999761 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 30 01:07:42.999761 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 30 01:07:42.999761 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 could not be found. Oct 30 01:07:42.999761 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:07:42.999761 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:07:42.999761 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:07:42.999761 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:07:42.999761 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:07:42.999761 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:07:42.999761 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 01:07:42.999761 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 01:07:42.999761 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 01:07:43.001506 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 01:07:43.001506 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 30 01:07:43.001506 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 30 01:07:43.001506 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 30 01:07:43.001506 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 30 01:07:43.001506 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 30 01:07:43.001506 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 30 01:07:43.001506 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 30 01:07:43.001506 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 30 01:07:43.001506 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 could not be found. Oct 30 01:07:43.001506 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:07:43.001506 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:07:43.001506 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:07:43.001506 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:07:43.001506 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:07:43.001506 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:07:43.001506 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 01:07:43.001506 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 01:07:43.001506 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 could not be found. Oct 30 01:07:43.001506 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:07:43.003046 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-80ee9beb-0f82-4aba-a876-d7a5fdc8307f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Returning 404 to user: Instance 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 could not be found. {{(pid=118961) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 01:07:43.003046 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-80ee9beb-0f82-4aba-a876-d7a5fdc8307f tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "DELETE /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 404 len: 111 microversion: 2.1 time: 0.031083 Oct 30 01:07:43.003046 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 621/1217] 10.222.0.16 () {58 vars in 1239 bytes} [Sun Oct 30 01:07:42 2022] DELETE /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 30 01:07:43.006649 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-2d55ae00-0770-42eb-b8fa-2c4a25fb08d1 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:07:43.007260 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-2d55ae00-0770-42eb-b8fa-2c4a25fb08d1 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] [instance: 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:07:43.013968 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2d55ae00-0770-42eb-b8fa-2c4a25fb08d1 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:43.014475 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2d55ae00-0770-42eb-b8fa-2c4a25fb08d1 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:43.014865 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2d55ae00-0770-42eb-b8fa-2c4a25fb08d1 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:43.015566 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2d55ae00-0770-42eb-b8fa-2c4a25fb08d1 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:43.016062 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2d55ae00-0770-42eb-b8fa-2c4a25fb08d1 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:43.016473 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2d55ae00-0770-42eb-b8fa-2c4a25fb08d1 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:43.030532 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.wsgi [None req-2d55ae00-0770-42eb-b8fa-2c4a25fb08d1 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] HTTP exception thrown: Instance 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 could not be found. Oct 30 01:07:43.032062 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-2d55ae00-0770-42eb-b8fa-2c4a25fb08d1 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Request method failure captured: Oct 30 01:07:43.032062 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: request: GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 HTTP/1.1 Oct 30 01:07:43.032062 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept: application/json Oct 30 01:07:43.032062 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept-Encoding: identity Oct 30 01:07:43.032062 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Connection: close Oct 30 01:07:43.032062 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Length: 0 Oct 30 01:07:43.032062 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Type: application/json Oct 30 01:07:43.032062 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Host: 10.222.0.16 Oct 30 01:07:43.032062 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Openstack-System-Scope: None Oct 30 01:07:43.032062 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: User-Agent: python-urllib3/1.26.12 Oct 30 01:07:43.032062 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Auth-Token: gAAAAABjXcufJywtdFM5PObVwvc4XfvnADFZy4zohCugYTCT6kU4WYIm9tEeU-k9LqQ38owLu40IDCuTtQtYT30Plcr8MKntgwthetV3Q2djsPe7ErJwj43_xWiRbN29t1NWovB4v2ZaEkG-yf_zjlv7AkC5WmwDZXliUm7DodcvaVfundKFDhc Oct 30 01:07:43.032062 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Id: None Oct 30 01:07:43.032062 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Name: None Oct 30 01:07:43.032062 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Identity-Status: Confirmed Oct 30 01:07:43.032062 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Is-Admin-Project: True Oct 30 01:07:43.032062 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Id: default Oct 30 01:07:43.032062 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Name: Default Oct 30 01:07:43.032062 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Id: 8bf407954ed34ebfaf56db206df4fc84 Oct 30 01:07:43.032062 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Name: tempest-AttachVolumeTestJSON-298540627 Oct 30 01:07:43.032062 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Role: reader,member Oct 30 01:07:43.032062 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Roles: reader,member Oct 30 01:07:43.033663 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_8bf407954ed34ebfaf56db206df4fc84"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8bf407954ed34ebfaf56db206df4fc84"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/8bf407954ed34ebfaf56db206df4fc84"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8bf407954ed34ebfaf56db206df4fc84"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 01:07:43.033663 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant: tempest-AttachVolumeTestJSON-298540627 Oct 30 01:07:43.033663 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Id: 8bf407954ed34ebfaf56db206df4fc84 Oct 30 01:07:43.033663 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Name: tempest-AttachVolumeTestJSON-298540627 Oct 30 01:07:43.033663 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User: tempest-AttachVolumeTestJSON-298540627-project Oct 30 01:07:43.033663 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Id: default Oct 30 01:07:43.033663 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Name: Default Oct 30 01:07:43.033663 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Id: 1c0e5bd11ac14259ad376ce54bcb7025 Oct 30 01:07:43.033663 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Name: tempest-AttachVolumeTestJSON-298540627-project Oct 30 01:07:43.033663 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: method: > Oct 30 01:07:43.033663 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: exception: Instance 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 could not be found. Oct 30 01:07:43.033663 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: action_args: {'id': '70ad3ec1-64bc-40ac-bb98-cc82d19e2a14'} Oct 30 01:07:43.034532 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 01:07:43.034532 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:07:43.034532 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 30 01:07:43.034532 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 30 01:07:43.034532 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 30 01:07:43.034532 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 30 01:07:43.034532 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 30 01:07:43.034532 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 30 01:07:43.034532 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 30 01:07:43.034532 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 30 01:07:43.034532 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 could not be found. Oct 30 01:07:43.034532 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:07:43.034532 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:07:43.034532 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:07:43.034532 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:07:43.034532 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:07:43.034532 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:07:43.034532 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 01:07:43.034532 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 01:07:43.034532 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 01:07:43.036097 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 01:07:43.036097 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 30 01:07:43.036097 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 30 01:07:43.036097 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 30 01:07:43.036097 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 30 01:07:43.036097 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 30 01:07:43.036097 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 30 01:07:43.036097 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 could not be found. Oct 30 01:07:43.036097 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:07:43.036097 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:07:43.036097 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:07:43.036097 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:07:43.036097 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:07:43.036097 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:07:43.036097 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 01:07:43.036097 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 01:07:43.036097 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 could not be found. Oct 30 01:07:43.036097 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:07:43.037692 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-2d55ae00-0770-42eb-b8fa-2c4a25fb08d1 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] Returning 404 to user: Instance 70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 could not be found. {{(pid=118962) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 01:07:43.037692 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-2d55ae00-0770-42eb-b8fa-2c4a25fb08d1 tempest-AttachVolumeTestJSON-298540627 tempest-AttachVolumeTestJSON-298540627-project] 10.222.0.16 "GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14" status: 404 len: 111 microversion: 2.1 time: 0.030789 Oct 30 01:07:43.037692 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 597/1218] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:07:43 2022] GET /compute/v2.1/servers/70ad3ec1-64bc-40ac-bb98-cc82d19e2a14 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 30 01:07:50.512228 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-2e687deb-3bf6-46af-b2f3-acf208cf518e tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-21067465", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1884997751", "security_groups": [{"name": "tempest-secgroup-smoke-1108501088"}], "block_device_mapping_v2": [{"uuid": "54171fe1-01cf-43fe-9e0f-62297f055415", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "c1a88451-f287-4e35-b7e8-a6d4e4aaa603"}]}} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:07:50.633002 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.network.neutron [None req-2e687deb-3bf6-46af-b2f3-acf208cf518e tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] validate_networks() for [('c1a88451-f287-4e35-b7e8-a6d4e4aaa603', None, None, None, None, None)] {{(pid=118961) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 30 01:07:51.120532 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-2e687deb-3bf6-46af-b2f3-acf208cf518e tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Flavor limits 0:0:0 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 30 01:07:51.121144 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-2e687deb-3bf6-46af-b2f3-acf208cf518e tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Image limits 0:0:0 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 30 01:07:51.121689 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-2e687deb-3bf6-46af-b2f3-acf208cf518e tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Flavor pref 0:0:0 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 30 01:07:51.122354 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-2e687deb-3bf6-46af-b2f3-acf208cf518e tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Image pref 0:0:0 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 30 01:07:51.123544 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-2e687deb-3bf6-46af-b2f3-acf208cf518e tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 30 01:07:51.135543 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2e687deb-3bf6-46af-b2f3-acf208cf518e tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:51.136152 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2e687deb-3bf6-46af-b2f3-acf208cf518e tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:51.136862 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2e687deb-3bf6-46af-b2f3-acf208cf518e tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:51.137686 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2e687deb-3bf6-46af-b2f3-acf208cf518e tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:51.138350 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2e687deb-3bf6-46af-b2f3-acf208cf518e tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:51.138845 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2e687deb-3bf6-46af-b2f3-acf208cf518e tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:51.157958 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.quota [None req-2e687deb-3bf6-46af-b2f3-acf208cf518e tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Getting quotas for project 238d71be4e094cb6b84f9a48a8a2c79c. Resources: {'ram', 'instances', 'cores'} {{(pid=118961) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 30 01:07:51.162786 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.quota [None req-2e687deb-3bf6-46af-b2f3-acf208cf518e tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Getting quotas for user 96e0cd00e4ef4a2da960aa5d4f66972a and project 238d71be4e094cb6b84f9a48a8a2c79c. Resources: {'ram', 'instances', 'cores'} {{(pid=118961) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 30 01:07:51.221002 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-2e687deb-3bf6-46af-b2f3-acf208cf518e tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Going to run 1 instances... {{(pid=118961) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 30 01:07:51.223194 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-2e687deb-3bf6-46af-b2f3-acf208cf518e tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: 91a8549d-4f75-4c98-9773-98efe4196889] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='54171fe1-01cf-43fe-9e0f-62297f055415',volume_size=None,volume_type=None)] {{(pid=118961) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 30 01:07:51.363879 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR oslo.messaging._drivers.impl_rabbit [None req-2e687deb-3bf6-46af-b2f3-acf208cf518e tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [b36edcff-6649-43d9-a913-81fa9ddad5ca] AMQP server on 10.222.0.16:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Oct 30 01:07:52.373357 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO oslo.messaging._drivers.impl_rabbit [None req-2e687deb-3bf6-46af-b2f3-acf208cf518e tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [b36edcff-6649-43d9-a913-81fa9ddad5ca] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 60838. Oct 30 01:07:52.389678 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-2e687deb-3bf6-46af-b2f3-acf208cf518e tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 1.881336 Oct 30 01:07:52.390311 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 622/1219] 10.222.0.16 () {60 vars in 1147 bytes} [Sun Oct 30 01:07:50 2022] POST /compute/v2.1/servers => generated 406 bytes in 1883 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 30 01:07:52.403896 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-39b77ee4-08aa-4c67-a243-079a18b5f648 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:07:52.406024 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-39b77ee4-08aa-4c67-a243-079a18b5f648 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: 91a8549d-4f75-4c98-9773-98efe4196889] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:07:52.463261 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-39b77ee4-08aa-4c67-a243-079a18b5f648 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:07:52.470261 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-39b77ee4-08aa-4c67-a243-079a18b5f648 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/91a8549d-4f75-4c98-9773-98efe4196889" status: 200 len: 1093 microversion: 2.1 time: 0.072582 Oct 30 01:07:52.470827 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 598/1220] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:07:52 2022] GET /compute/v2.1/servers/91a8549d-4f75-4c98-9773-98efe4196889 => generated 1093 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:07:53.494250 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-46e0275e-9379-4871-954c-6f16d24febbe tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:07:53.494903 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-46e0275e-9379-4871-954c-6f16d24febbe tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: 91a8549d-4f75-4c98-9773-98efe4196889] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:07:53.501636 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-46e0275e-9379-4871-954c-6f16d24febbe tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:53.502305 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-46e0275e-9379-4871-954c-6f16d24febbe tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:53.502744 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-46e0275e-9379-4871-954c-6f16d24febbe tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:53.503447 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-46e0275e-9379-4871-954c-6f16d24febbe tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:53.503953 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-46e0275e-9379-4871-954c-6f16d24febbe tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:53.504403 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-46e0275e-9379-4871-954c-6f16d24febbe tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:53.508505 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell1:SELECT=52,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:07:53.577614 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-46e0275e-9379-4871-954c-6f16d24febbe tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:07:53.587107 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-46e0275e-9379-4871-954c-6f16d24febbe tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/91a8549d-4f75-4c98-9773-98efe4196889" status: 200 len: 1191 microversion: 2.1 time: 0.097703 Oct 30 01:07:53.588457 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 623/1221] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:07:53 2022] GET /compute/v2.1/servers/91a8549d-4f75-4c98-9773-98efe4196889 => generated 1191 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:07:54.610450 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-a3a1f792-9e19-42cd-adab-e5cdad186dda tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:07:54.611124 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-a3a1f792-9e19-42cd-adab-e5cdad186dda tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: 91a8549d-4f75-4c98-9773-98efe4196889] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:07:54.620654 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell1:SELECT=42,nova_cell1:INSERT=2 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:07:54.623982 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a3a1f792-9e19-42cd-adab-e5cdad186dda tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:54.625127 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a3a1f792-9e19-42cd-adab-e5cdad186dda tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:54.625716 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a3a1f792-9e19-42cd-adab-e5cdad186dda tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:54.626599 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a3a1f792-9e19-42cd-adab-e5cdad186dda tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:54.627403 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a3a1f792-9e19-42cd-adab-e5cdad186dda tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:54.628011 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a3a1f792-9e19-42cd-adab-e5cdad186dda tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:54.769578 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-a3a1f792-9e19-42cd-adab-e5cdad186dda tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:07:54.778303 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-a3a1f792-9e19-42cd-adab-e5cdad186dda tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/91a8549d-4f75-4c98-9773-98efe4196889" status: 200 len: 1277 microversion: 2.1 time: 0.171811 Oct 30 01:07:54.778977 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 599/1222] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:07:54 2022] GET /compute/v2.1/servers/91a8549d-4f75-4c98-9773-98efe4196889 => generated 1277 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:07:55.062115 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [req-033fc8fb-1d4a-4b84-b9e5-69e5833a1907 req-2a6847cd-83e2-49c6-95cf-1a8b55d06ccb service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "91a8549d-4f75-4c98-9773-98efe4196889", "tag": "7cefd299-3b60-4011-b5ad-6ad6926ac49d"}]} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:07:55.071549 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-033fc8fb-1d4a-4b84-b9e5-69e5833a1907 req-2a6847cd-83e2-49c6-95cf-1a8b55d06ccb service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:55.072047 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-033fc8fb-1d4a-4b84-b9e5-69e5833a1907 req-2a6847cd-83e2-49c6-95cf-1a8b55d06ccb service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:55.072465 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-033fc8fb-1d4a-4b84-b9e5-69e5833a1907 req-2a6847cd-83e2-49c6-95cf-1a8b55d06ccb service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:55.088021 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.compute.server_external_events [req-033fc8fb-1d4a-4b84-b9e5-69e5833a1907 req-2a6847cd-83e2-49c6-95cf-1a8b55d06ccb service nova] Creating event network-changed:7cefd299-3b60-4011-b5ad-6ad6926ac49d for instance 91a8549d-4f75-4c98-9773-98efe4196889 on hv-cinder-maste Oct 30 01:07:55.093397 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [req-033fc8fb-1d4a-4b84-b9e5-69e5833a1907 req-2a6847cd-83e2-49c6-95cf-1a8b55d06ccb service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035722 Oct 30 01:07:55.093995 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 624/1223] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:07:55 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:07:55.801535 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-5f5d8d4d-9f33-4f22-b06f-62d6c48a2bb1 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:07:55.802145 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-5f5d8d4d-9f33-4f22-b06f-62d6c48a2bb1 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: 91a8549d-4f75-4c98-9773-98efe4196889] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:07:55.808712 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5f5d8d4d-9f33-4f22-b06f-62d6c48a2bb1 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:55.809199 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5f5d8d4d-9f33-4f22-b06f-62d6c48a2bb1 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:55.809631 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5f5d8d4d-9f33-4f22-b06f-62d6c48a2bb1 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:55.810367 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5f5d8d4d-9f33-4f22-b06f-62d6c48a2bb1 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:55.810816 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5f5d8d4d-9f33-4f22-b06f-62d6c48a2bb1 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:55.811236 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5f5d8d4d-9f33-4f22-b06f-62d6c48a2bb1 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:55.946666 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-5f5d8d4d-9f33-4f22-b06f-62d6c48a2bb1 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:07:55.954503 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-5f5d8d4d-9f33-4f22-b06f-62d6c48a2bb1 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/91a8549d-4f75-4c98-9773-98efe4196889" status: 200 len: 1265 microversion: 2.1 time: 0.157038 Oct 30 01:07:55.955135 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 600/1224] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:07:55 2022] GET /compute/v2.1/servers/91a8549d-4f75-4c98-9773-98efe4196889 => generated 1265 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:07:56.983859 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-89464c9d-42f7-473f-bea9-e7d6c07f69a5 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:07:56.985028 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-89464c9d-42f7-473f-bea9-e7d6c07f69a5 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: 91a8549d-4f75-4c98-9773-98efe4196889] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:07:56.992005 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-89464c9d-42f7-473f-bea9-e7d6c07f69a5 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:56.992484 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-89464c9d-42f7-473f-bea9-e7d6c07f69a5 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:56.992910 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-89464c9d-42f7-473f-bea9-e7d6c07f69a5 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:56.993594 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-89464c9d-42f7-473f-bea9-e7d6c07f69a5 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:56.994022 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-89464c9d-42f7-473f-bea9-e7d6c07f69a5 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:56.994450 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-89464c9d-42f7-473f-bea9-e7d6c07f69a5 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:57.131596 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-89464c9d-42f7-473f-bea9-e7d6c07f69a5 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:07:57.143476 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-89464c9d-42f7-473f-bea9-e7d6c07f69a5 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/91a8549d-4f75-4c98-9773-98efe4196889" status: 200 len: 1265 microversion: 2.1 time: 0.167221 Oct 30 01:07:57.144283 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 625/1225] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:07:56 2022] GET /compute/v2.1/servers/91a8549d-4f75-4c98-9773-98efe4196889 => generated 1265 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:07:58.175783 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-432a3b75-c23f-4298-85f8-80ab51fdadd5 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:07:58.176764 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-432a3b75-c23f-4298-85f8-80ab51fdadd5 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: 91a8549d-4f75-4c98-9773-98efe4196889] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:07:58.187646 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-432a3b75-c23f-4298-85f8-80ab51fdadd5 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:58.188518 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-432a3b75-c23f-4298-85f8-80ab51fdadd5 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:58.189319 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-432a3b75-c23f-4298-85f8-80ab51fdadd5 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:58.190109 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-432a3b75-c23f-4298-85f8-80ab51fdadd5 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:58.190559 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-432a3b75-c23f-4298-85f8-80ab51fdadd5 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:58.190982 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-432a3b75-c23f-4298-85f8-80ab51fdadd5 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:58.337609 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-432a3b75-c23f-4298-85f8-80ab51fdadd5 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:07:58.346764 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-432a3b75-c23f-4298-85f8-80ab51fdadd5 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/91a8549d-4f75-4c98-9773-98efe4196889" status: 200 len: 1265 microversion: 2.1 time: 0.177563 Oct 30 01:07:58.347373 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 601/1226] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:07:58 2022] GET /compute/v2.1/servers/91a8549d-4f75-4c98-9773-98efe4196889 => generated 1265 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:07:59.374738 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-d9415e11-3fd2-4038-91da-325bb1e8eef8 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:07:59.375816 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-d9415e11-3fd2-4038-91da-325bb1e8eef8 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: 91a8549d-4f75-4c98-9773-98efe4196889] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:07:59.383917 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d9415e11-3fd2-4038-91da-325bb1e8eef8 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:59.384400 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d9415e11-3fd2-4038-91da-325bb1e8eef8 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:59.384838 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d9415e11-3fd2-4038-91da-325bb1e8eef8 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:59.385569 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d9415e11-3fd2-4038-91da-325bb1e8eef8 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:07:59.386021 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d9415e11-3fd2-4038-91da-325bb1e8eef8 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:07:59.386440 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d9415e11-3fd2-4038-91da-325bb1e8eef8 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:07:59.524735 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-d9415e11-3fd2-4038-91da-325bb1e8eef8 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:07:59.532786 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-d9415e11-3fd2-4038-91da-325bb1e8eef8 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/91a8549d-4f75-4c98-9773-98efe4196889" status: 200 len: 1265 microversion: 2.1 time: 0.162763 Oct 30 01:07:59.533419 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 626/1227] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:07:59 2022] GET /compute/v2.1/servers/91a8549d-4f75-4c98-9773-98efe4196889 => generated 1265 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:08:00.555660 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-824045bc-d34d-4ef2-9bd2-eb15640215e6 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:08:00.556258 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-824045bc-d34d-4ef2-9bd2-eb15640215e6 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: 91a8549d-4f75-4c98-9773-98efe4196889] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:08:00.562765 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-824045bc-d34d-4ef2-9bd2-eb15640215e6 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:00.563473 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-824045bc-d34d-4ef2-9bd2-eb15640215e6 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:00.564283 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-824045bc-d34d-4ef2-9bd2-eb15640215e6 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:00.565204 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-824045bc-d34d-4ef2-9bd2-eb15640215e6 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:00.565705 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-824045bc-d34d-4ef2-9bd2-eb15640215e6 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:00.566136 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-824045bc-d34d-4ef2-9bd2-eb15640215e6 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:00.698827 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-824045bc-d34d-4ef2-9bd2-eb15640215e6 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:08:00.707003 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-824045bc-d34d-4ef2-9bd2-eb15640215e6 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/91a8549d-4f75-4c98-9773-98efe4196889" status: 200 len: 1265 microversion: 2.1 time: 0.155306 Oct 30 01:08:00.707696 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 602/1228] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:08:00 2022] GET /compute/v2.1/servers/91a8549d-4f75-4c98-9773-98efe4196889 => generated 1265 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:08:01.732501 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-c158f30e-f3a6-4e9b-92fb-120271c15264 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:08:01.733120 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-c158f30e-f3a6-4e9b-92fb-120271c15264 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: 91a8549d-4f75-4c98-9773-98efe4196889] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:08:01.735653 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell0:SELECT=2 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:08:01.736445 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell0:SELECT=6 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:08:01.742989 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c158f30e-f3a6-4e9b-92fb-120271c15264 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:01.743480 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c158f30e-f3a6-4e9b-92fb-120271c15264 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:01.743950 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c158f30e-f3a6-4e9b-92fb-120271c15264 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:01.744649 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c158f30e-f3a6-4e9b-92fb-120271c15264 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:01.745147 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c158f30e-f3a6-4e9b-92fb-120271c15264 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:01.745783 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-c158f30e-f3a6-4e9b-92fb-120271c15264 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:01.877927 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-c158f30e-f3a6-4e9b-92fb-120271c15264 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:08:01.886412 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-c158f30e-f3a6-4e9b-92fb-120271c15264 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/91a8549d-4f75-4c98-9773-98efe4196889" status: 200 len: 1265 microversion: 2.1 time: 0.157840 Oct 30 01:08:01.887015 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 627/1229] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:08:01 2022] GET /compute/v2.1/servers/91a8549d-4f75-4c98-9773-98efe4196889 => generated 1265 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:08:02.909631 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-244c5858-f764-45d7-b9cb-9ce523ecdcdb tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:08:02.910299 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-244c5858-f764-45d7-b9cb-9ce523ecdcdb tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: 91a8549d-4f75-4c98-9773-98efe4196889] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:08:02.912456 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell0:SELECT=2 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:08:02.913107 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_api:SELECT=29,nova_api:DELETE=1 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:08:02.918811 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-244c5858-f764-45d7-b9cb-9ce523ecdcdb tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:02.919298 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-244c5858-f764-45d7-b9cb-9ce523ecdcdb tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:02.919758 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-244c5858-f764-45d7-b9cb-9ce523ecdcdb tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:02.920464 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-244c5858-f764-45d7-b9cb-9ce523ecdcdb tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:02.920914 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-244c5858-f764-45d7-b9cb-9ce523ecdcdb tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:02.921386 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-244c5858-f764-45d7-b9cb-9ce523ecdcdb tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:03.071198 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-244c5858-f764-45d7-b9cb-9ce523ecdcdb tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:08:03.080396 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-244c5858-f764-45d7-b9cb-9ce523ecdcdb tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/91a8549d-4f75-4c98-9773-98efe4196889" status: 200 len: 1265 microversion: 2.1 time: 0.175132 Oct 30 01:08:03.081164 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 603/1230] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:08:02 2022] GET /compute/v2.1/servers/91a8549d-4f75-4c98-9773-98efe4196889 => generated 1265 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:08:04.104396 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-cc619173-7a70-4f7f-bf7e-8362de24ee4f tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:08:04.105040 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-cc619173-7a70-4f7f-bf7e-8362de24ee4f tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: 91a8549d-4f75-4c98-9773-98efe4196889] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:08:04.112401 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cc619173-7a70-4f7f-bf7e-8362de24ee4f tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:04.112894 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cc619173-7a70-4f7f-bf7e-8362de24ee4f tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:04.113318 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cc619173-7a70-4f7f-bf7e-8362de24ee4f tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:04.114035 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cc619173-7a70-4f7f-bf7e-8362de24ee4f tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:04.114488 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cc619173-7a70-4f7f-bf7e-8362de24ee4f tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:04.114919 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-cc619173-7a70-4f7f-bf7e-8362de24ee4f tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:04.250431 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-cc619173-7a70-4f7f-bf7e-8362de24ee4f tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:08:04.258861 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-cc619173-7a70-4f7f-bf7e-8362de24ee4f tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/91a8549d-4f75-4c98-9773-98efe4196889" status: 200 len: 1265 microversion: 2.1 time: 0.158361 Oct 30 01:08:04.259407 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 628/1231] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:08:04 2022] GET /compute/v2.1/servers/91a8549d-4f75-4c98-9773-98efe4196889 => generated 1265 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:08:05.285939 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-591c4666-6ac7-4321-9a5c-bf482ae4952d tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:08:05.286944 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-591c4666-6ac7-4321-9a5c-bf482ae4952d tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: 91a8549d-4f75-4c98-9773-98efe4196889] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:08:05.298404 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-591c4666-6ac7-4321-9a5c-bf482ae4952d tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:05.299047 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-591c4666-6ac7-4321-9a5c-bf482ae4952d tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:05.299476 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-591c4666-6ac7-4321-9a5c-bf482ae4952d tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:05.300206 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-591c4666-6ac7-4321-9a5c-bf482ae4952d tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:05.300658 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-591c4666-6ac7-4321-9a5c-bf482ae4952d tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:05.301074 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-591c4666-6ac7-4321-9a5c-bf482ae4952d tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:05.442278 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-591c4666-6ac7-4321-9a5c-bf482ae4952d tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:08:05.452529 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-591c4666-6ac7-4321-9a5c-bf482ae4952d tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/91a8549d-4f75-4c98-9773-98efe4196889" status: 200 len: 1265 microversion: 2.1 time: 0.173269 Oct 30 01:08:05.453117 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 604/1232] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:08:05 2022] GET /compute/v2.1/servers/91a8549d-4f75-4c98-9773-98efe4196889 => generated 1265 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:08:06.476654 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-63c7ee42-e002-4c27-b417-2642ed4d9c2f tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:08:06.477352 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-63c7ee42-e002-4c27-b417-2642ed4d9c2f tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: 91a8549d-4f75-4c98-9773-98efe4196889] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:08:06.484757 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-63c7ee42-e002-4c27-b417-2642ed4d9c2f tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:06.485453 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-63c7ee42-e002-4c27-b417-2642ed4d9c2f tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:06.486054 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-63c7ee42-e002-4c27-b417-2642ed4d9c2f tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:06.486852 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-63c7ee42-e002-4c27-b417-2642ed4d9c2f tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:06.487379 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-63c7ee42-e002-4c27-b417-2642ed4d9c2f tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:06.487832 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-63c7ee42-e002-4c27-b417-2642ed4d9c2f tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:06.634209 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-63c7ee42-e002-4c27-b417-2642ed4d9c2f tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:08:06.642014 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-63c7ee42-e002-4c27-b417-2642ed4d9c2f tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/91a8549d-4f75-4c98-9773-98efe4196889" status: 200 len: 1265 microversion: 2.1 time: 0.169572 Oct 30 01:08:06.642779 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 629/1233] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:08:06 2022] GET /compute/v2.1/servers/91a8549d-4f75-4c98-9773-98efe4196889 => generated 1265 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:08:07.110853 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [req-a2e895a1-a8b3-408e-9164-4af6f9812eff req-7fa3f5a8-bbaa-4262-ad87-6eb0097563da service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "91a8549d-4f75-4c98-9773-98efe4196889", "name": "network-vif-plugged", "status": "completed", "tag": "7cefd299-3b60-4011-b5ad-6ad6926ac49d"}]} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:08:07.121352 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-a2e895a1-a8b3-408e-9164-4af6f9812eff req-7fa3f5a8-bbaa-4262-ad87-6eb0097563da service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:07.121936 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-a2e895a1-a8b3-408e-9164-4af6f9812eff req-7fa3f5a8-bbaa-4262-ad87-6eb0097563da service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:07.122357 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-a2e895a1-a8b3-408e-9164-4af6f9812eff req-7fa3f5a8-bbaa-4262-ad87-6eb0097563da service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:07.134441 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.compute.server_external_events [req-a2e895a1-a8b3-408e-9164-4af6f9812eff req-7fa3f5a8-bbaa-4262-ad87-6eb0097563da service nova] Creating event network-vif-plugged:7cefd299-3b60-4011-b5ad-6ad6926ac49d for instance 91a8549d-4f75-4c98-9773-98efe4196889 on hv-cinder-maste Oct 30 01:08:07.140571 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [req-a2e895a1-a8b3-408e-9164-4af6f9812eff req-7fa3f5a8-bbaa-4262-ad87-6eb0097563da service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.220940 Oct 30 01:08:07.141108 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 605/1234] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:08:06 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 30 01:08:07.664935 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-ec9ca470-a5d5-4dc0-b666-9528fac03078 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:08:07.665450 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-ec9ca470-a5d5-4dc0-b666-9528fac03078 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: 91a8549d-4f75-4c98-9773-98efe4196889] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:08:07.671929 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ec9ca470-a5d5-4dc0-b666-9528fac03078 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:07.672444 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ec9ca470-a5d5-4dc0-b666-9528fac03078 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:07.672875 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ec9ca470-a5d5-4dc0-b666-9528fac03078 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:07.673585 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ec9ca470-a5d5-4dc0-b666-9528fac03078 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:07.674030 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ec9ca470-a5d5-4dc0-b666-9528fac03078 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:07.674442 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ec9ca470-a5d5-4dc0-b666-9528fac03078 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:07.809612 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-ec9ca470-a5d5-4dc0-b666-9528fac03078 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:08:07.817746 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-ec9ca470-a5d5-4dc0-b666-9528fac03078 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/91a8549d-4f75-4c98-9773-98efe4196889" status: 200 len: 1265 microversion: 2.1 time: 0.156700 Oct 30 01:08:07.818348 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 630/1235] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:08:07 2022] GET /compute/v2.1/servers/91a8549d-4f75-4c98-9773-98efe4196889 => generated 1265 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:08:08.841588 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-178c7333-8c66-4076-a878-1447e5728414 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:08:08.842286 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-178c7333-8c66-4076-a878-1447e5728414 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: 91a8549d-4f75-4c98-9773-98efe4196889] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:08:08.848914 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-178c7333-8c66-4076-a878-1447e5728414 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:08.849379 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-178c7333-8c66-4076-a878-1447e5728414 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:08.849802 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-178c7333-8c66-4076-a878-1447e5728414 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:08.850496 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-178c7333-8c66-4076-a878-1447e5728414 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:08.850938 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-178c7333-8c66-4076-a878-1447e5728414 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:08.851332 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-178c7333-8c66-4076-a878-1447e5728414 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:08.992163 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-178c7333-8c66-4076-a878-1447e5728414 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:08:09.000780 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-178c7333-8c66-4076-a878-1447e5728414 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/91a8549d-4f75-4c98-9773-98efe4196889" status: 200 len: 1265 microversion: 2.1 time: 0.164277 Oct 30 01:08:09.001420 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 606/1236] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:08:08 2022] GET /compute/v2.1/servers/91a8549d-4f75-4c98-9773-98efe4196889 => generated 1265 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:08:10.023644 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-da747921-b0cf-4e46-bbf5-977d52231b21 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:08:10.024316 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-da747921-b0cf-4e46-bbf5-977d52231b21 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: 91a8549d-4f75-4c98-9773-98efe4196889] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:08:10.026626 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_api:SELECT=39,nova_api:INSERT=3 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:08:10.032904 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-da747921-b0cf-4e46-bbf5-977d52231b21 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:10.033636 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-da747921-b0cf-4e46-bbf5-977d52231b21 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:10.034227 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-da747921-b0cf-4e46-bbf5-977d52231b21 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:10.035101 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-da747921-b0cf-4e46-bbf5-977d52231b21 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:10.035705 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-da747921-b0cf-4e46-bbf5-977d52231b21 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:10.036317 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-da747921-b0cf-4e46-bbf5-977d52231b21 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:10.191413 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-da747921-b0cf-4e46-bbf5-977d52231b21 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:08:10.200601 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-da747921-b0cf-4e46-bbf5-977d52231b21 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/91a8549d-4f75-4c98-9773-98efe4196889" status: 200 len: 1453 microversion: 2.1 time: 0.180592 Oct 30 01:08:10.201285 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 631/1237] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:08:10 2022] GET /compute/v2.1/servers/91a8549d-4f75-4c98-9773-98efe4196889 => generated 1453 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:08:10.224399 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-5f39a622-bb3d-4570-bea6-f64c86069723 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:08:10.225143 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-5f39a622-bb3d-4570-bea6-f64c86069723 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: 91a8549d-4f75-4c98-9773-98efe4196889] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:08:10.231883 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5f39a622-bb3d-4570-bea6-f64c86069723 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:10.232426 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5f39a622-bb3d-4570-bea6-f64c86069723 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:10.232855 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5f39a622-bb3d-4570-bea6-f64c86069723 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:10.233693 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5f39a622-bb3d-4570-bea6-f64c86069723 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:10.234186 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5f39a622-bb3d-4570-bea6-f64c86069723 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:10.234632 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5f39a622-bb3d-4570-bea6-f64c86069723 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:10.377029 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-5f39a622-bb3d-4570-bea6-f64c86069723 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:08:10.384953 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-5f39a622-bb3d-4570-bea6-f64c86069723 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/91a8549d-4f75-4c98-9773-98efe4196889" status: 200 len: 1453 microversion: 2.1 time: 0.164674 Oct 30 01:08:10.385499 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 607/1238] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:08:10 2022] GET /compute/v2.1/servers/91a8549d-4f75-4c98-9773-98efe4196889 => generated 1453 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:08:12.505617 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [req-bcad3b0b-c171-4c13-93ae-f4c15cd7da23 req-1a4d18e5-63e4-450b-ab03-f522981886bc service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "91a8549d-4f75-4c98-9773-98efe4196889", "tag": "7cefd299-3b60-4011-b5ad-6ad6926ac49d"}]} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:08:12.516216 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-bcad3b0b-c171-4c13-93ae-f4c15cd7da23 req-1a4d18e5-63e4-450b-ab03-f522981886bc service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:12.516874 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-bcad3b0b-c171-4c13-93ae-f4c15cd7da23 req-1a4d18e5-63e4-450b-ab03-f522981886bc service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:12.517348 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-bcad3b0b-c171-4c13-93ae-f4c15cd7da23 req-1a4d18e5-63e4-450b-ab03-f522981886bc service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:12.529099 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.compute.server_external_events [req-bcad3b0b-c171-4c13-93ae-f4c15cd7da23 req-1a4d18e5-63e4-450b-ab03-f522981886bc service nova] Creating event network-changed:7cefd299-3b60-4011-b5ad-6ad6926ac49d for instance 91a8549d-4f75-4c98-9773-98efe4196889 on hv-cinder-maste Oct 30 01:08:12.534434 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [req-bcad3b0b-c171-4c13-93ae-f4c15cd7da23 req-1a4d18e5-63e4-450b-ab03-f522981886bc service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033209 Oct 30 01:08:12.535122 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 632/1239] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:08:12 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:08:13.617200 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-4b10c696-c489-4d90-9c7d-e736c5e0d22a tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:08:13.617726 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-4b10c696-c489-4d90-9c7d-e736c5e0d22a tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: ffdd5acc-f9cf-4244-ae3b-429260b7c33f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:08:13.624491 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4b10c696-c489-4d90-9c7d-e736c5e0d22a tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:13.624973 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4b10c696-c489-4d90-9c7d-e736c5e0d22a tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:13.625409 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4b10c696-c489-4d90-9c7d-e736c5e0d22a tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:13.626292 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4b10c696-c489-4d90-9c7d-e736c5e0d22a tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:13.626741 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4b10c696-c489-4d90-9c7d-e736c5e0d22a tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:13.627199 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4b10c696-c489-4d90-9c7d-e736c5e0d22a tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:13.658977 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-4b10c696-c489-4d90-9c7d-e736c5e0d22a tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/ffdd5acc-f9cf-4244-ae3b-429260b7c33f/ips" status: 200 len: 148 microversion: 2.1 time: 0.045844 Oct 30 01:08:13.659548 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 608/1240] 10.222.0.16 () {58 vars in 1248 bytes} [Sun Oct 30 01:08:13 2022] GET /compute/v2.1/servers/ffdd5acc-f9cf-4244-ae3b-429260b7c33f/ips => generated 148 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:08:15.029257 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-462f9aaa-53ab-43e3-be1b-eac5d81b6fbe tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "5f3c61bb-1509-41b6-b1f8-8b3d244349e4"}} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:08:15.031794 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-462f9aaa-53ab-43e3-be1b-eac5d81b6fbe tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: ffdd5acc-f9cf-4244-ae3b-429260b7c33f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:08:15.038647 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-462f9aaa-53ab-43e3-be1b-eac5d81b6fbe tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:15.039137 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-462f9aaa-53ab-43e3-be1b-eac5d81b6fbe tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:15.039577 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-462f9aaa-53ab-43e3-be1b-eac5d81b6fbe tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:15.040320 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-462f9aaa-53ab-43e3-be1b-eac5d81b6fbe tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:15.040764 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-462f9aaa-53ab-43e3-be1b-eac5d81b6fbe tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:15.041196 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-462f9aaa-53ab-43e3-be1b-eac5d81b6fbe tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:15.331748 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-462f9aaa-53ab-43e3-be1b-eac5d81b6fbe tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "POST /compute/v2.1/servers/ffdd5acc-f9cf-4244-ae3b-429260b7c33f/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.307402 Oct 30 01:08:15.332480 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 633/1241] 10.222.0.16 () {60 vars in 1323 bytes} [Sun Oct 30 01:08:15 2022] POST /compute/v2.1/servers/ffdd5acc-f9cf-4244-ae3b-429260b7c33f/os-volume_attachments => generated 194 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:08:23.013079 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-8ef00cc6-260d-4d65-ab7a-bb8af6260ee2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:08:23.013672 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-8ef00cc6-260d-4d65-ab7a-bb8af6260ee2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: ffdd5acc-f9cf-4244-ae3b-429260b7c33f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:08:23.020609 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8ef00cc6-260d-4d65-ab7a-bb8af6260ee2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:23.021097 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8ef00cc6-260d-4d65-ab7a-bb8af6260ee2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:23.021529 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8ef00cc6-260d-4d65-ab7a-bb8af6260ee2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:23.022475 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8ef00cc6-260d-4d65-ab7a-bb8af6260ee2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:23.023065 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8ef00cc6-260d-4d65-ab7a-bb8af6260ee2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:23.023490 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8ef00cc6-260d-4d65-ab7a-bb8af6260ee2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:23.109977 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.objects.instance [None req-8ef00cc6-260d-4d65-ab7a-bb8af6260ee2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lazy-loading 'services' on Instance uuid ffdd5acc-f9cf-4244-ae3b-429260b7c33f {{(pid=118962) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 30 01:08:23.188969 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-8ef00cc6-260d-4d65-ab7a-bb8af6260ee2 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "DELETE /compute/v2.1/servers/ffdd5acc-f9cf-4244-ae3b-429260b7c33f/os-volume_attachments/5f3c61bb-1509-41b6-b1f8-8b3d244349e4" status: 202 len: 0 microversion: 2.1 time: 0.179810 Oct 30 01:08:23.189509 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 609/1242] 10.222.0.16 () {58 vars in 1416 bytes} [Sun Oct 30 01:08:23 2022] DELETE /compute/v2.1/servers/ffdd5acc-f9cf-4244-ae3b-429260b7c33f/os-volume_attachments/5f3c61bb-1509-41b6-b1f8-8b3d244349e4 => generated 0 bytes in 181 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 30 01:08:23.200554 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-01713c40-c5ed-4f74-9bff-fad95cc1857f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:08:23.203041 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-01713c40-c5ed-4f74-9bff-fad95cc1857f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: ffdd5acc-f9cf-4244-ae3b-429260b7c33f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:08:23.210058 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-01713c40-c5ed-4f74-9bff-fad95cc1857f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:23.210605 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-01713c40-c5ed-4f74-9bff-fad95cc1857f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:23.211383 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-01713c40-c5ed-4f74-9bff-fad95cc1857f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:23.212203 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-01713c40-c5ed-4f74-9bff-fad95cc1857f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:23.212682 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-01713c40-c5ed-4f74-9bff-fad95cc1857f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:23.213068 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-01713c40-c5ed-4f74-9bff-fad95cc1857f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:23.246352 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-01713c40-c5ed-4f74-9bff-fad95cc1857f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/ffdd5acc-f9cf-4244-ae3b-429260b7c33f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051678 Oct 30 01:08:23.246996 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 634/1243] 10.222.0.16 () {58 vars in 1302 bytes} [Sun Oct 30 01:08:23 2022] GET /compute/v2.1/servers/ffdd5acc-f9cf-4244-ae3b-429260b7c33f/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:08:24.260437 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-24fa8098-e3dd-4c36-968f-5a4dec6216fb tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:08:24.262727 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-24fa8098-e3dd-4c36-968f-5a4dec6216fb tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: ffdd5acc-f9cf-4244-ae3b-429260b7c33f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:08:24.269113 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-24fa8098-e3dd-4c36-968f-5a4dec6216fb tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:24.269585 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-24fa8098-e3dd-4c36-968f-5a4dec6216fb tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:24.270008 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-24fa8098-e3dd-4c36-968f-5a4dec6216fb tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:24.270694 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-24fa8098-e3dd-4c36-968f-5a4dec6216fb tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:24.271132 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-24fa8098-e3dd-4c36-968f-5a4dec6216fb tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:24.271540 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-24fa8098-e3dd-4c36-968f-5a4dec6216fb tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:24.302052 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-24fa8098-e3dd-4c36-968f-5a4dec6216fb tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/ffdd5acc-f9cf-4244-ae3b-429260b7c33f/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.046922 Oct 30 01:08:24.302615 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 610/1244] 10.222.0.16 () {58 vars in 1302 bytes} [Sun Oct 30 01:08:24 2022] GET /compute/v2.1/servers/ffdd5acc-f9cf-4244-ae3b-429260b7c33f/os-volume_attachments => generated 25 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 30 01:08:24.387348 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-6b8659e1-747a-4452-bff8-cc3083f8a04a tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:08:24.387944 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-6b8659e1-747a-4452-bff8-cc3083f8a04a tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: ffdd5acc-f9cf-4244-ae3b-429260b7c33f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:08:24.398931 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6b8659e1-747a-4452-bff8-cc3083f8a04a tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:24.399458 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6b8659e1-747a-4452-bff8-cc3083f8a04a tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:24.400267 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6b8659e1-747a-4452-bff8-cc3083f8a04a tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:24.401430 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6b8659e1-747a-4452-bff8-cc3083f8a04a tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:24.402147 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6b8659e1-747a-4452-bff8-cc3083f8a04a tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:24.402791 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6b8659e1-747a-4452-bff8-cc3083f8a04a tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:24.406702 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell1:SELECT=69,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:08:24.441598 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-6b8659e1-747a-4452-bff8-cc3083f8a04a tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: ffdd5acc-f9cf-4244-ae3b-429260b7c33f] Going to try to terminate instance {{(pid=118961) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 30 01:08:24.531706 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-6b8659e1-747a-4452-bff8-cc3083f8a04a tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "DELETE /compute/v2.1/servers/ffdd5acc-f9cf-4244-ae3b-429260b7c33f" status: 204 len: 0 microversion: 2.1 time: 0.148599 Oct 30 01:08:24.532294 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 635/1245] 10.222.0.16 () {58 vars in 1239 bytes} [Sun Oct 30 01:08:24 2022] DELETE /compute/v2.1/servers/ffdd5acc-f9cf-4244-ae3b-429260b7c33f => generated 0 bytes in 150 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 30 01:08:24.540609 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-602c1a8e-c46b-4661-9383-604511bb533f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:08:24.541217 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-602c1a8e-c46b-4661-9383-604511bb533f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: ffdd5acc-f9cf-4244-ae3b-429260b7c33f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:08:24.547908 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-602c1a8e-c46b-4661-9383-604511bb533f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:24.548393 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-602c1a8e-c46b-4661-9383-604511bb533f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:24.548828 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-602c1a8e-c46b-4661-9383-604511bb533f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:24.549517 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-602c1a8e-c46b-4661-9383-604511bb533f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:24.549971 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-602c1a8e-c46b-4661-9383-604511bb533f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:24.550390 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-602c1a8e-c46b-4661-9383-604511bb533f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:24.702930 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-602c1a8e-c46b-4661-9383-604511bb533f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:08:24.705417 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell1:SELECT=74,nova_cell1:INSERT=1 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:08:24.713030 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-602c1a8e-c46b-4661-9383-604511bb533f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/ffdd5acc-f9cf-4244-ae3b-429260b7c33f" status: 200 len: 1680 microversion: 2.1 time: 0.176353 Oct 30 01:08:24.713619 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 611/1246] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:08:24 2022] GET /compute/v2.1/servers/ffdd5acc-f9cf-4244-ae3b-429260b7c33f => generated 1680 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:08:25.736429 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-d73e7fe6-aad7-4022-81f2-de4a4b4a4ee9 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:08:25.737143 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-d73e7fe6-aad7-4022-81f2-de4a4b4a4ee9 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: ffdd5acc-f9cf-4244-ae3b-429260b7c33f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:08:25.743861 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d73e7fe6-aad7-4022-81f2-de4a4b4a4ee9 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:25.744343 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d73e7fe6-aad7-4022-81f2-de4a4b4a4ee9 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:25.744777 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d73e7fe6-aad7-4022-81f2-de4a4b4a4ee9 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:25.745465 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d73e7fe6-aad7-4022-81f2-de4a4b4a4ee9 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:25.745906 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d73e7fe6-aad7-4022-81f2-de4a4b4a4ee9 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:25.746320 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d73e7fe6-aad7-4022-81f2-de4a4b4a4ee9 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:25.882464 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-d73e7fe6-aad7-4022-81f2-de4a4b4a4ee9 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:08:25.890725 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-d73e7fe6-aad7-4022-81f2-de4a4b4a4ee9 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/ffdd5acc-f9cf-4244-ae3b-429260b7c33f" status: 200 len: 1680 microversion: 2.1 time: 0.158556 Oct 30 01:08:25.891303 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 636/1247] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:08:25 2022] GET /compute/v2.1/servers/ffdd5acc-f9cf-4244-ae3b-429260b7c33f => generated 1680 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:08:26.914511 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-d4ee4d13-0fb0-4489-b96b-225e5534e5a5 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:08:26.915093 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-d4ee4d13-0fb0-4489-b96b-225e5534e5a5 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: ffdd5acc-f9cf-4244-ae3b-429260b7c33f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:08:26.921924 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d4ee4d13-0fb0-4489-b96b-225e5534e5a5 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:26.922412 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d4ee4d13-0fb0-4489-b96b-225e5534e5a5 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:26.922884 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d4ee4d13-0fb0-4489-b96b-225e5534e5a5 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:26.923611 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d4ee4d13-0fb0-4489-b96b-225e5534e5a5 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:26.924266 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d4ee4d13-0fb0-4489-b96b-225e5534e5a5 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:26.924712 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d4ee4d13-0fb0-4489-b96b-225e5534e5a5 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:27.067074 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-d4ee4d13-0fb0-4489-b96b-225e5534e5a5 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:08:27.076644 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-d4ee4d13-0fb0-4489-b96b-225e5534e5a5 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/ffdd5acc-f9cf-4244-ae3b-429260b7c33f" status: 200 len: 1680 microversion: 2.1 time: 0.165988 Oct 30 01:08:27.077459 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 612/1248] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:08:26 2022] GET /compute/v2.1/servers/ffdd5acc-f9cf-4244-ae3b-429260b7c33f => generated 1680 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:08:27.783124 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [req-4a3bc773-df70-4465-898b-d665a5bd2abb req-d8d4b885-5ea3-44cd-85a6-5344dc277461 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ffdd5acc-f9cf-4244-ae3b-429260b7c33f", "name": "network-vif-deleted", "tag": "c36e47a4-5169-485f-88ec-96caef1c3d0f"}]} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:08:27.799450 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-4a3bc773-df70-4465-898b-d665a5bd2abb req-d8d4b885-5ea3-44cd-85a6-5344dc277461 service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:27.800328 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-4a3bc773-df70-4465-898b-d665a5bd2abb req-d8d4b885-5ea3-44cd-85a6-5344dc277461 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:27.800888 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-4a3bc773-df70-4465-898b-d665a5bd2abb req-d8d4b885-5ea3-44cd-85a6-5344dc277461 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:27.820425 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.compute.server_external_events [req-4a3bc773-df70-4465-898b-d665a5bd2abb req-d8d4b885-5ea3-44cd-85a6-5344dc277461 service nova] Creating event network-vif-deleted:c36e47a4-5169-485f-88ec-96caef1c3d0f for instance ffdd5acc-f9cf-4244-ae3b-429260b7c33f on hv-cinder-maste Oct 30 01:08:27.826415 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [req-4a3bc773-df70-4465-898b-d665a5bd2abb req-d8d4b885-5ea3-44cd-85a6-5344dc277461 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.046590 Oct 30 01:08:27.827743 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 637/1249] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:08:27 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:08:28.101432 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-2efd2261-3f70-47f5-a0c9-e7502fa5d60c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:08:28.102774 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-2efd2261-3f70-47f5-a0c9-e7502fa5d60c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: ffdd5acc-f9cf-4244-ae3b-429260b7c33f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:08:28.109914 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2efd2261-3f70-47f5-a0c9-e7502fa5d60c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:28.110604 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2efd2261-3f70-47f5-a0c9-e7502fa5d60c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:28.111059 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2efd2261-3f70-47f5-a0c9-e7502fa5d60c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:28.111753 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2efd2261-3f70-47f5-a0c9-e7502fa5d60c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:28.112202 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2efd2261-3f70-47f5-a0c9-e7502fa5d60c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:28.112796 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2efd2261-3f70-47f5-a0c9-e7502fa5d60c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:28.144879 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2efd2261-3f70-47f5-a0c9-e7502fa5d60c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:28.145355 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2efd2261-3f70-47f5-a0c9-e7502fa5d60c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:28.145784 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2efd2261-3f70-47f5-a0c9-e7502fa5d60c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:28.146230 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.objects.instance [None req-2efd2261-3f70-47f5-a0c9-e7502fa5d60c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lazy-loading 'fault' on Instance uuid ffdd5acc-f9cf-4244-ae3b-429260b7c33f {{(pid=118962) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 30 01:08:28.186300 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-2efd2261-3f70-47f5-a0c9-e7502fa5d60c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ecaf8ec105c04b919256f852da859eba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b68b886c19c45d585409651f33b8626', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:08:28.195017 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-2efd2261-3f70-47f5-a0c9-e7502fa5d60c tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/ffdd5acc-f9cf-4244-ae3b-429260b7c33f" status: 200 len: 1331 microversion: 2.1 time: 0.097960 Oct 30 01:08:28.195631 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 613/1250] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:08:28 2022] GET /compute/v2.1/servers/ffdd5acc-f9cf-4244-ae3b-429260b7c33f => generated 1331 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:08:29.226346 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-b50ae121-8462-4d04-9256-f45a1a6d3e41 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:08:29.227155 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-b50ae121-8462-4d04-9256-f45a1a6d3e41 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: ffdd5acc-f9cf-4244-ae3b-429260b7c33f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:08:29.234745 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b50ae121-8462-4d04-9256-f45a1a6d3e41 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:29.235224 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b50ae121-8462-4d04-9256-f45a1a6d3e41 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:29.235661 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b50ae121-8462-4d04-9256-f45a1a6d3e41 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:29.236375 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b50ae121-8462-4d04-9256-f45a1a6d3e41 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:29.236820 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b50ae121-8462-4d04-9256-f45a1a6d3e41 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:29.237356 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b50ae121-8462-4d04-9256-f45a1a6d3e41 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:29.250782 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.wsgi [None req-b50ae121-8462-4d04-9256-f45a1a6d3e41 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] HTTP exception thrown: Instance ffdd5acc-f9cf-4244-ae3b-429260b7c33f could not be found. Oct 30 01:08:29.252550 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-b50ae121-8462-4d04-9256-f45a1a6d3e41 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Request method failure captured: Oct 30 01:08:29.252550 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: request: GET /compute/v2.1/servers/ffdd5acc-f9cf-4244-ae3b-429260b7c33f HTTP/1.1 Oct 30 01:08:29.252550 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Accept: application/json Oct 30 01:08:29.252550 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Accept-Encoding: identity Oct 30 01:08:29.252550 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Connection: close Oct 30 01:08:29.252550 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Content-Length: 0 Oct 30 01:08:29.252550 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Content-Type: application/json Oct 30 01:08:29.252550 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Host: 10.222.0.16 Oct 30 01:08:29.252550 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Openstack-System-Scope: None Oct 30 01:08:29.252550 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: User-Agent: python-urllib3/1.26.12 Oct 30 01:08:29.252550 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Auth-Token: gAAAAABjXcuedsaRjtnPRcHo4HqTnifOYe1vcHHjSzl7t8SsEKfOt41JBob3t9Ien606ucEZmcgmpf5H0lzlL6oZS4vLpZERLur9y2YRhqd9AYPyhbrmUkrYmKAssv6PSCysf6z3-myn1YKr62Nxqfs7t1lZGZnZuH_I2j8LHl5PnVXWJxyADCY Oct 30 01:08:29.252550 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Domain-Id: None Oct 30 01:08:29.252550 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Domain-Name: None Oct 30 01:08:29.252550 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Identity-Status: Confirmed Oct 30 01:08:29.252550 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Is-Admin-Project: True Oct 30 01:08:29.252550 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Domain-Id: default Oct 30 01:08:29.252550 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Domain-Name: Default Oct 30 01:08:29.252550 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Id: 6b68b886c19c45d585409651f33b8626 Oct 30 01:08:29.252550 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Name: tempest-AttachVolumeNegativeTest-618009204 Oct 30 01:08:29.252550 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Role: reader,member Oct 30 01:08:29.252550 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Roles: reader,member Oct 30 01:08:29.254202 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_6b68b886c19c45d585409651f33b8626"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6b68b886c19c45d585409651f33b8626"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/6b68b886c19c45d585409651f33b8626"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6b68b886c19c45d585409651f33b8626"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 01:08:29.254202 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant: tempest-AttachVolumeNegativeTest-618009204 Oct 30 01:08:29.254202 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant-Id: 6b68b886c19c45d585409651f33b8626 Oct 30 01:08:29.254202 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-618009204 Oct 30 01:08:29.254202 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User: tempest-AttachVolumeNegativeTest-618009204-project Oct 30 01:08:29.254202 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Domain-Id: default Oct 30 01:08:29.254202 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Domain-Name: Default Oct 30 01:08:29.254202 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Id: ecaf8ec105c04b919256f852da859eba Oct 30 01:08:29.254202 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Name: tempest-AttachVolumeNegativeTest-618009204-project Oct 30 01:08:29.254202 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: method: > Oct 30 01:08:29.254202 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: exception: Instance ffdd5acc-f9cf-4244-ae3b-429260b7c33f could not be found. Oct 30 01:08:29.254202 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: action_args: {'id': 'ffdd5acc-f9cf-4244-ae3b-429260b7c33f'} Oct 30 01:08:29.255150 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 01:08:29.255150 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:29.255150 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 30 01:08:29.255150 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 30 01:08:29.255150 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 30 01:08:29.255150 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 30 01:08:29.255150 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 30 01:08:29.255150 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 30 01:08:29.255150 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 30 01:08:29.255150 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 30 01:08:29.255150 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ffdd5acc-f9cf-4244-ae3b-429260b7c33f could not be found. Oct 30 01:08:29.255150 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:08:29.255150 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:08:29.255150 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:08:29.255150 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:29.255150 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:08:29.255150 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:08:29.255150 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 01:08:29.255150 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 01:08:29.255150 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 01:08:29.256879 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 01:08:29.256879 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 30 01:08:29.256879 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 30 01:08:29.256879 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 30 01:08:29.256879 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 30 01:08:29.256879 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 30 01:08:29.256879 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 30 01:08:29.256879 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ffdd5acc-f9cf-4244-ae3b-429260b7c33f could not be found. Oct 30 01:08:29.256879 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:08:29.256879 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:08:29.256879 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:08:29.256879 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:29.256879 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:08:29.256879 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:08:29.256879 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 01:08:29.256879 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 01:08:29.256879 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ffdd5acc-f9cf-4244-ae3b-429260b7c33f could not be found. Oct 30 01:08:29.256879 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:08:29.258299 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-b50ae121-8462-4d04-9256-f45a1a6d3e41 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Returning 404 to user: Instance ffdd5acc-f9cf-4244-ae3b-429260b7c33f could not be found. {{(pid=118961) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 01:08:29.258299 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-b50ae121-8462-4d04-9256-f45a1a6d3e41 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/ffdd5acc-f9cf-4244-ae3b-429260b7c33f" status: 404 len: 111 microversion: 2.1 time: 0.035862 Oct 30 01:08:29.258299 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 638/1251] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:08:29 2022] GET /compute/v2.1/servers/ffdd5acc-f9cf-4244-ae3b-429260b7c33f => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 30 01:08:29.263097 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-c5511b2e-151d-42ed-a39b-3810288230fc tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '.version_select at 0x7f0eafd80dc0>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:08:29.284971 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-c5511b2e-151d-42ed-a39b-3810288230fc tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-741020447" status: 202 len: 0 microversion: 2.1 time: 0.025959 Oct 30 01:08:29.285552 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 614/1252] 10.222.0.16 () {58 vars in 1218 bytes} [Sun Oct 30 01:08:29 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-741020447 => generated 0 bytes in 27 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 30 01:08:31.089381 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-23ea2103-b176-4e45-8365-8644362daec5 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:08:31.090107 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-23ea2103-b176-4e45-8365-8644362daec5 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: ffdd5acc-f9cf-4244-ae3b-429260b7c33f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:08:31.096690 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-23ea2103-b176-4e45-8365-8644362daec5 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:31.097177 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-23ea2103-b176-4e45-8365-8644362daec5 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:31.097596 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-23ea2103-b176-4e45-8365-8644362daec5 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:31.098318 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-23ea2103-b176-4e45-8365-8644362daec5 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:31.098776 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-23ea2103-b176-4e45-8365-8644362daec5 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:31.099179 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-23ea2103-b176-4e45-8365-8644362daec5 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:31.115797 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.wsgi [None req-23ea2103-b176-4e45-8365-8644362daec5 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] HTTP exception thrown: Instance ffdd5acc-f9cf-4244-ae3b-429260b7c33f could not be found. Oct 30 01:08:31.117344 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-23ea2103-b176-4e45-8365-8644362daec5 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Request method failure captured: Oct 30 01:08:31.117344 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: request: DELETE /compute/v2.1/servers/ffdd5acc-f9cf-4244-ae3b-429260b7c33f HTTP/1.1 Oct 30 01:08:31.117344 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Accept: application/json Oct 30 01:08:31.117344 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Accept-Encoding: identity Oct 30 01:08:31.117344 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Connection: close Oct 30 01:08:31.117344 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Content-Length: 0 Oct 30 01:08:31.117344 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Content-Type: application/json Oct 30 01:08:31.117344 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Host: 10.222.0.16 Oct 30 01:08:31.117344 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Openstack-System-Scope: None Oct 30 01:08:31.117344 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: User-Agent: python-urllib3/1.26.12 Oct 30 01:08:31.117344 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Auth-Token: gAAAAABjXcuedsaRjtnPRcHo4HqTnifOYe1vcHHjSzl7t8SsEKfOt41JBob3t9Ien606ucEZmcgmpf5H0lzlL6oZS4vLpZERLur9y2YRhqd9AYPyhbrmUkrYmKAssv6PSCysf6z3-myn1YKr62Nxqfs7t1lZGZnZuH_I2j8LHl5PnVXWJxyADCY Oct 30 01:08:31.117344 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Domain-Id: None Oct 30 01:08:31.117344 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Domain-Name: None Oct 30 01:08:31.117344 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Identity-Status: Confirmed Oct 30 01:08:31.117344 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Is-Admin-Project: True Oct 30 01:08:31.117344 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Domain-Id: default Oct 30 01:08:31.117344 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Domain-Name: Default Oct 30 01:08:31.117344 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Id: 6b68b886c19c45d585409651f33b8626 Oct 30 01:08:31.117344 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Name: tempest-AttachVolumeNegativeTest-618009204 Oct 30 01:08:31.117344 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Role: reader,member Oct 30 01:08:31.117344 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Roles: reader,member Oct 30 01:08:31.119061 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_6b68b886c19c45d585409651f33b8626"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6b68b886c19c45d585409651f33b8626"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/6b68b886c19c45d585409651f33b8626"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6b68b886c19c45d585409651f33b8626"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 01:08:31.119061 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant: tempest-AttachVolumeNegativeTest-618009204 Oct 30 01:08:31.119061 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant-Id: 6b68b886c19c45d585409651f33b8626 Oct 30 01:08:31.119061 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-618009204 Oct 30 01:08:31.119061 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User: tempest-AttachVolumeNegativeTest-618009204-project Oct 30 01:08:31.119061 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Domain-Id: default Oct 30 01:08:31.119061 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Domain-Name: Default Oct 30 01:08:31.119061 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Id: ecaf8ec105c04b919256f852da859eba Oct 30 01:08:31.119061 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Name: tempest-AttachVolumeNegativeTest-618009204-project Oct 30 01:08:31.119061 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: method: > Oct 30 01:08:31.119061 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: exception: Instance ffdd5acc-f9cf-4244-ae3b-429260b7c33f could not be found. Oct 30 01:08:31.119061 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: action_args: {'id': 'ffdd5acc-f9cf-4244-ae3b-429260b7c33f'} Oct 30 01:08:31.120004 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 01:08:31.120004 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:31.120004 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 30 01:08:31.120004 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 30 01:08:31.120004 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 30 01:08:31.120004 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 30 01:08:31.120004 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 30 01:08:31.120004 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 30 01:08:31.120004 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 30 01:08:31.120004 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 30 01:08:31.120004 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ffdd5acc-f9cf-4244-ae3b-429260b7c33f could not be found. Oct 30 01:08:31.120004 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:08:31.120004 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:08:31.120004 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:08:31.120004 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:31.120004 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:08:31.120004 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:08:31.120004 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 01:08:31.120004 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 01:08:31.120004 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 01:08:31.121672 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 01:08:31.121672 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 30 01:08:31.121672 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 30 01:08:31.121672 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 30 01:08:31.121672 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 30 01:08:31.121672 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 30 01:08:31.121672 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 30 01:08:31.121672 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 30 01:08:31.121672 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 30 01:08:31.121672 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ffdd5acc-f9cf-4244-ae3b-429260b7c33f could not be found. Oct 30 01:08:31.121672 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:08:31.121672 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:08:31.121672 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:08:31.121672 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:31.121672 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:08:31.121672 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:08:31.121672 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 01:08:31.121672 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 01:08:31.121672 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ffdd5acc-f9cf-4244-ae3b-429260b7c33f could not be found. Oct 30 01:08:31.121672 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:08:31.123978 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-23ea2103-b176-4e45-8365-8644362daec5 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Returning 404 to user: Instance ffdd5acc-f9cf-4244-ae3b-429260b7c33f could not be found. {{(pid=118961) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 01:08:31.124826 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-23ea2103-b176-4e45-8365-8644362daec5 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "DELETE /compute/v2.1/servers/ffdd5acc-f9cf-4244-ae3b-429260b7c33f" status: 404 len: 111 microversion: 2.1 time: 0.042022 Oct 30 01:08:31.125384 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 639/1253] 10.222.0.16 () {58 vars in 1239 bytes} [Sun Oct 30 01:08:31 2022] DELETE /compute/v2.1/servers/ffdd5acc-f9cf-4244-ae3b-429260b7c33f => generated 111 bytes in 43 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 30 01:08:31.134234 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-3141db24-2b74-495f-ae42-d6ec82a961b3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:08:31.134861 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-3141db24-2b74-495f-ae42-d6ec82a961b3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: ffdd5acc-f9cf-4244-ae3b-429260b7c33f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:08:31.141960 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3141db24-2b74-495f-ae42-d6ec82a961b3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:31.142456 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3141db24-2b74-495f-ae42-d6ec82a961b3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:31.142911 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3141db24-2b74-495f-ae42-d6ec82a961b3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:31.143589 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3141db24-2b74-495f-ae42-d6ec82a961b3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:31.144100 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3141db24-2b74-495f-ae42-d6ec82a961b3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:31.144602 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3141db24-2b74-495f-ae42-d6ec82a961b3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:31.158927 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.wsgi [None req-3141db24-2b74-495f-ae42-d6ec82a961b3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] HTTP exception thrown: Instance ffdd5acc-f9cf-4244-ae3b-429260b7c33f could not be found. Oct 30 01:08:31.160670 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-3141db24-2b74-495f-ae42-d6ec82a961b3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Request method failure captured: Oct 30 01:08:31.160670 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: request: GET /compute/v2.1/servers/ffdd5acc-f9cf-4244-ae3b-429260b7c33f HTTP/1.1 Oct 30 01:08:31.160670 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept: application/json Oct 30 01:08:31.160670 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept-Encoding: identity Oct 30 01:08:31.160670 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Connection: close Oct 30 01:08:31.160670 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Length: 0 Oct 30 01:08:31.160670 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Type: application/json Oct 30 01:08:31.160670 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Host: 10.222.0.16 Oct 30 01:08:31.160670 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Openstack-System-Scope: None Oct 30 01:08:31.160670 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: User-Agent: python-urllib3/1.26.12 Oct 30 01:08:31.160670 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Auth-Token: gAAAAABjXcuedsaRjtnPRcHo4HqTnifOYe1vcHHjSzl7t8SsEKfOt41JBob3t9Ien606ucEZmcgmpf5H0lzlL6oZS4vLpZERLur9y2YRhqd9AYPyhbrmUkrYmKAssv6PSCysf6z3-myn1YKr62Nxqfs7t1lZGZnZuH_I2j8LHl5PnVXWJxyADCY Oct 30 01:08:31.160670 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Id: None Oct 30 01:08:31.160670 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Name: None Oct 30 01:08:31.160670 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Identity-Status: Confirmed Oct 30 01:08:31.160670 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Is-Admin-Project: True Oct 30 01:08:31.160670 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Id: default Oct 30 01:08:31.160670 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Name: Default Oct 30 01:08:31.160670 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Id: 6b68b886c19c45d585409651f33b8626 Oct 30 01:08:31.160670 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Name: tempest-AttachVolumeNegativeTest-618009204 Oct 30 01:08:31.160670 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Role: reader,member Oct 30 01:08:31.160670 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Roles: reader,member Oct 30 01:08:31.162295 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_6b68b886c19c45d585409651f33b8626"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6b68b886c19c45d585409651f33b8626"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/6b68b886c19c45d585409651f33b8626"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6b68b886c19c45d585409651f33b8626"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 01:08:31.162295 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant: tempest-AttachVolumeNegativeTest-618009204 Oct 30 01:08:31.162295 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Id: 6b68b886c19c45d585409651f33b8626 Oct 30 01:08:31.162295 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-618009204 Oct 30 01:08:31.162295 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User: tempest-AttachVolumeNegativeTest-618009204-project Oct 30 01:08:31.162295 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Id: default Oct 30 01:08:31.162295 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Name: Default Oct 30 01:08:31.162295 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Id: ecaf8ec105c04b919256f852da859eba Oct 30 01:08:31.162295 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Name: tempest-AttachVolumeNegativeTest-618009204-project Oct 30 01:08:31.162295 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: method: > Oct 30 01:08:31.162295 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: exception: Instance ffdd5acc-f9cf-4244-ae3b-429260b7c33f could not be found. Oct 30 01:08:31.162295 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: action_args: {'id': 'ffdd5acc-f9cf-4244-ae3b-429260b7c33f'} Oct 30 01:08:31.166287 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 01:08:31.166287 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:31.166287 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 30 01:08:31.166287 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 30 01:08:31.166287 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 30 01:08:31.166287 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 30 01:08:31.166287 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 30 01:08:31.166287 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 30 01:08:31.166287 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 30 01:08:31.166287 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 30 01:08:31.166287 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ffdd5acc-f9cf-4244-ae3b-429260b7c33f could not be found. Oct 30 01:08:31.166287 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:08:31.166287 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:08:31.166287 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:08:31.166287 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:31.166287 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:08:31.166287 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:08:31.166287 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 01:08:31.166287 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 01:08:31.166287 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 01:08:31.167985 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 01:08:31.167985 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 30 01:08:31.167985 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 30 01:08:31.167985 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 30 01:08:31.167985 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 30 01:08:31.167985 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 30 01:08:31.167985 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 30 01:08:31.167985 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ffdd5acc-f9cf-4244-ae3b-429260b7c33f could not be found. Oct 30 01:08:31.167985 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:08:31.167985 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:08:31.167985 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:08:31.167985 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:31.167985 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:08:31.167985 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:08:31.167985 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 01:08:31.167985 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 01:08:31.167985 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ffdd5acc-f9cf-4244-ae3b-429260b7c33f could not be found. Oct 30 01:08:31.167985 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:08:31.170197 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-3141db24-2b74-495f-ae42-d6ec82a961b3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Returning 404 to user: Instance ffdd5acc-f9cf-4244-ae3b-429260b7c33f could not be found. {{(pid=118962) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 01:08:31.170197 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-3141db24-2b74-495f-ae42-d6ec82a961b3 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/ffdd5acc-f9cf-4244-ae3b-429260b7c33f" status: 404 len: 111 microversion: 2.1 time: 0.032491 Oct 30 01:08:31.170197 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 615/1254] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:08:31 2022] GET /compute/v2.1/servers/ffdd5acc-f9cf-4244-ae3b-429260b7c33f => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 30 01:08:32.317078 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-b5667ec3-7b7e-4cb4-add3-a35196316a9f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:08:32.317576 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-b5667ec3-7b7e-4cb4-add3-a35196316a9f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: b77a1841-4f87-4b8a-96a9-a619ea67d452] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:08:32.326862 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b5667ec3-7b7e-4cb4-add3-a35196316a9f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:32.327350 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b5667ec3-7b7e-4cb4-add3-a35196316a9f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:32.327889 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b5667ec3-7b7e-4cb4-add3-a35196316a9f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:32.328599 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b5667ec3-7b7e-4cb4-add3-a35196316a9f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:32.329082 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b5667ec3-7b7e-4cb4-add3-a35196316a9f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:32.329510 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b5667ec3-7b7e-4cb4-add3-a35196316a9f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:32.344102 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.wsgi [None req-b5667ec3-7b7e-4cb4-add3-a35196316a9f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] HTTP exception thrown: Instance b77a1841-4f87-4b8a-96a9-a619ea67d452 could not be found. Oct 30 01:08:32.345721 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-b5667ec3-7b7e-4cb4-add3-a35196316a9f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Request method failure captured: Oct 30 01:08:32.345721 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: request: DELETE /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452 HTTP/1.1 Oct 30 01:08:32.345721 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Accept: application/json Oct 30 01:08:32.345721 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Accept-Encoding: identity Oct 30 01:08:32.345721 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Connection: close Oct 30 01:08:32.345721 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Content-Length: 0 Oct 30 01:08:32.345721 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Content-Type: application/json Oct 30 01:08:32.345721 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Host: 10.222.0.16 Oct 30 01:08:32.345721 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Openstack-System-Scope: None Oct 30 01:08:32.345721 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: User-Agent: python-urllib3/1.26.12 Oct 30 01:08:32.345721 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Auth-Token: gAAAAABjXcuedsaRjtnPRcHo4HqTnifOYe1vcHHjSzl7t8SsEKfOt41JBob3t9Ien606ucEZmcgmpf5H0lzlL6oZS4vLpZERLur9y2YRhqd9AYPyhbrmUkrYmKAssv6PSCysf6z3-myn1YKr62Nxqfs7t1lZGZnZuH_I2j8LHl5PnVXWJxyADCY Oct 30 01:08:32.345721 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Domain-Id: None Oct 30 01:08:32.345721 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Domain-Name: None Oct 30 01:08:32.345721 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Identity-Status: Confirmed Oct 30 01:08:32.345721 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Is-Admin-Project: True Oct 30 01:08:32.345721 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Domain-Id: default Oct 30 01:08:32.345721 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Domain-Name: Default Oct 30 01:08:32.345721 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Id: 6b68b886c19c45d585409651f33b8626 Oct 30 01:08:32.345721 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Name: tempest-AttachVolumeNegativeTest-618009204 Oct 30 01:08:32.345721 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Role: reader,member Oct 30 01:08:32.345721 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Roles: reader,member Oct 30 01:08:32.347122 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_6b68b886c19c45d585409651f33b8626"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6b68b886c19c45d585409651f33b8626"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/6b68b886c19c45d585409651f33b8626"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6b68b886c19c45d585409651f33b8626"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 01:08:32.347122 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant: tempest-AttachVolumeNegativeTest-618009204 Oct 30 01:08:32.347122 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant-Id: 6b68b886c19c45d585409651f33b8626 Oct 30 01:08:32.347122 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-618009204 Oct 30 01:08:32.347122 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User: tempest-AttachVolumeNegativeTest-618009204-project Oct 30 01:08:32.347122 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Domain-Id: default Oct 30 01:08:32.347122 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Domain-Name: Default Oct 30 01:08:32.347122 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Id: ecaf8ec105c04b919256f852da859eba Oct 30 01:08:32.347122 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Name: tempest-AttachVolumeNegativeTest-618009204-project Oct 30 01:08:32.347122 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: method: > Oct 30 01:08:32.347122 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: exception: Instance b77a1841-4f87-4b8a-96a9-a619ea67d452 could not be found. Oct 30 01:08:32.347122 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: action_args: {'id': 'b77a1841-4f87-4b8a-96a9-a619ea67d452'} Oct 30 01:08:32.348249 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 01:08:32.348249 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:32.348249 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 30 01:08:32.348249 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 30 01:08:32.348249 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 30 01:08:32.348249 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 30 01:08:32.348249 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 30 01:08:32.348249 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 30 01:08:32.348249 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 30 01:08:32.348249 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 30 01:08:32.348249 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b77a1841-4f87-4b8a-96a9-a619ea67d452 could not be found. Oct 30 01:08:32.348249 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:08:32.348249 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:08:32.348249 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:08:32.348249 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:32.348249 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:08:32.348249 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:08:32.348249 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 01:08:32.348249 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 01:08:32.348249 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 01:08:32.350448 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 01:08:32.350448 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 30 01:08:32.350448 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 30 01:08:32.350448 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 30 01:08:32.350448 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 30 01:08:32.350448 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 30 01:08:32.350448 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 30 01:08:32.350448 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 30 01:08:32.350448 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 30 01:08:32.350448 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b77a1841-4f87-4b8a-96a9-a619ea67d452 could not be found. Oct 30 01:08:32.350448 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:08:32.350448 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:08:32.350448 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:08:32.350448 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:32.350448 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:08:32.350448 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:08:32.350448 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 01:08:32.350448 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 01:08:32.350448 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b77a1841-4f87-4b8a-96a9-a619ea67d452 could not be found. Oct 30 01:08:32.350448 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:08:32.352154 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-b5667ec3-7b7e-4cb4-add3-a35196316a9f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Returning 404 to user: Instance b77a1841-4f87-4b8a-96a9-a619ea67d452 could not be found. {{(pid=118961) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 01:08:32.352154 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-b5667ec3-7b7e-4cb4-add3-a35196316a9f tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "DELETE /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452" status: 404 len: 111 microversion: 2.1 time: 0.034020 Oct 30 01:08:32.352154 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 640/1255] 10.222.0.16 () {58 vars in 1239 bytes} [Sun Oct 30 01:08:32 2022] DELETE /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 30 01:08:32.356017 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-a080fd06-1b8e-41d1-a422-001177c1b295 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:08:32.356633 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-a080fd06-1b8e-41d1-a422-001177c1b295 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: b77a1841-4f87-4b8a-96a9-a619ea67d452] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:08:32.363093 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a080fd06-1b8e-41d1-a422-001177c1b295 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:32.363573 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a080fd06-1b8e-41d1-a422-001177c1b295 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:32.364037 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a080fd06-1b8e-41d1-a422-001177c1b295 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:32.364733 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a080fd06-1b8e-41d1-a422-001177c1b295 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:32.365167 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a080fd06-1b8e-41d1-a422-001177c1b295 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:32.365575 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a080fd06-1b8e-41d1-a422-001177c1b295 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:32.379165 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.wsgi [None req-a080fd06-1b8e-41d1-a422-001177c1b295 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] HTTP exception thrown: Instance b77a1841-4f87-4b8a-96a9-a619ea67d452 could not be found. Oct 30 01:08:32.380720 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-a080fd06-1b8e-41d1-a422-001177c1b295 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Request method failure captured: Oct 30 01:08:32.380720 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: request: GET /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452 HTTP/1.1 Oct 30 01:08:32.380720 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept: application/json Oct 30 01:08:32.380720 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept-Encoding: identity Oct 30 01:08:32.380720 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Connection: close Oct 30 01:08:32.380720 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Length: 0 Oct 30 01:08:32.380720 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Type: application/json Oct 30 01:08:32.380720 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Host: 10.222.0.16 Oct 30 01:08:32.380720 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Openstack-System-Scope: None Oct 30 01:08:32.380720 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: User-Agent: python-urllib3/1.26.12 Oct 30 01:08:32.380720 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Auth-Token: gAAAAABjXcuedsaRjtnPRcHo4HqTnifOYe1vcHHjSzl7t8SsEKfOt41JBob3t9Ien606ucEZmcgmpf5H0lzlL6oZS4vLpZERLur9y2YRhqd9AYPyhbrmUkrYmKAssv6PSCysf6z3-myn1YKr62Nxqfs7t1lZGZnZuH_I2j8LHl5PnVXWJxyADCY Oct 30 01:08:32.380720 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Id: None Oct 30 01:08:32.380720 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Name: None Oct 30 01:08:32.380720 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Identity-Status: Confirmed Oct 30 01:08:32.380720 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Is-Admin-Project: True Oct 30 01:08:32.380720 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Id: default Oct 30 01:08:32.380720 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Name: Default Oct 30 01:08:32.380720 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Id: 6b68b886c19c45d585409651f33b8626 Oct 30 01:08:32.380720 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Name: tempest-AttachVolumeNegativeTest-618009204 Oct 30 01:08:32.380720 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Role: reader,member Oct 30 01:08:32.380720 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Roles: reader,member Oct 30 01:08:32.382279 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_6b68b886c19c45d585409651f33b8626"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6b68b886c19c45d585409651f33b8626"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/6b68b886c19c45d585409651f33b8626"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6b68b886c19c45d585409651f33b8626"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 01:08:32.382279 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant: tempest-AttachVolumeNegativeTest-618009204 Oct 30 01:08:32.382279 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Id: 6b68b886c19c45d585409651f33b8626 Oct 30 01:08:32.382279 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-618009204 Oct 30 01:08:32.382279 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User: tempest-AttachVolumeNegativeTest-618009204-project Oct 30 01:08:32.382279 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Id: default Oct 30 01:08:32.382279 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Name: Default Oct 30 01:08:32.382279 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Id: ecaf8ec105c04b919256f852da859eba Oct 30 01:08:32.382279 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Name: tempest-AttachVolumeNegativeTest-618009204-project Oct 30 01:08:32.382279 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: method: > Oct 30 01:08:32.382279 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: exception: Instance b77a1841-4f87-4b8a-96a9-a619ea67d452 could not be found. Oct 30 01:08:32.382279 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: action_args: {'id': 'b77a1841-4f87-4b8a-96a9-a619ea67d452'} Oct 30 01:08:32.383119 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 01:08:32.383119 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:32.383119 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 30 01:08:32.383119 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 30 01:08:32.383119 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 30 01:08:32.383119 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 30 01:08:32.383119 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 30 01:08:32.383119 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 30 01:08:32.383119 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 30 01:08:32.383119 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 30 01:08:32.383119 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b77a1841-4f87-4b8a-96a9-a619ea67d452 could not be found. Oct 30 01:08:32.383119 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:08:32.383119 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:08:32.383119 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:08:32.383119 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:32.383119 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:08:32.383119 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:08:32.383119 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 01:08:32.383119 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 01:08:32.383119 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 01:08:32.384379 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 01:08:32.384379 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 30 01:08:32.384379 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 30 01:08:32.384379 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 30 01:08:32.384379 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 30 01:08:32.384379 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 30 01:08:32.384379 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 30 01:08:32.384379 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b77a1841-4f87-4b8a-96a9-a619ea67d452 could not be found. Oct 30 01:08:32.384379 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:08:32.384379 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:08:32.384379 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:08:32.384379 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:32.384379 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:08:32.384379 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:08:32.384379 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 01:08:32.384379 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 01:08:32.384379 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b77a1841-4f87-4b8a-96a9-a619ea67d452 could not be found. Oct 30 01:08:32.384379 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:08:32.385480 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-a080fd06-1b8e-41d1-a422-001177c1b295 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Returning 404 to user: Instance b77a1841-4f87-4b8a-96a9-a619ea67d452 could not be found. {{(pid=118962) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 01:08:32.385480 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-a080fd06-1b8e-41d1-a422-001177c1b295 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452" status: 404 len: 111 microversion: 2.1 time: 0.030067 Oct 30 01:08:32.385480 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 616/1256] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:08:32 2022] GET /compute/v2.1/servers/b77a1841-4f87-4b8a-96a9-a619ea67d452 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 30 01:08:32.391226 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-9f8088e6-ea24-460b-9ae3-d2db847464e1 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:08:32.391746 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-9f8088e6-ea24-460b-9ae3-d2db847464e1 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5632075c-7062-434e-b00c-bbf04c5ac04a] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:08:32.398292 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9f8088e6-ea24-460b-9ae3-d2db847464e1 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:32.398781 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9f8088e6-ea24-460b-9ae3-d2db847464e1 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:32.399215 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9f8088e6-ea24-460b-9ae3-d2db847464e1 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:32.399960 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9f8088e6-ea24-460b-9ae3-d2db847464e1 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:32.400418 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9f8088e6-ea24-460b-9ae3-d2db847464e1 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:32.400844 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9f8088e6-ea24-460b-9ae3-d2db847464e1 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:32.414500 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.wsgi [None req-9f8088e6-ea24-460b-9ae3-d2db847464e1 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] HTTP exception thrown: Instance 5632075c-7062-434e-b00c-bbf04c5ac04a could not be found. Oct 30 01:08:32.416116 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-9f8088e6-ea24-460b-9ae3-d2db847464e1 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Request method failure captured: Oct 30 01:08:32.416116 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: request: DELETE /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a HTTP/1.1 Oct 30 01:08:32.416116 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Accept: application/json Oct 30 01:08:32.416116 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Accept-Encoding: identity Oct 30 01:08:32.416116 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Connection: close Oct 30 01:08:32.416116 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Content-Length: 0 Oct 30 01:08:32.416116 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Content-Type: application/json Oct 30 01:08:32.416116 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Host: 10.222.0.16 Oct 30 01:08:32.416116 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Openstack-System-Scope: None Oct 30 01:08:32.416116 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: User-Agent: python-urllib3/1.26.12 Oct 30 01:08:32.416116 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Auth-Token: gAAAAABjXcuedsaRjtnPRcHo4HqTnifOYe1vcHHjSzl7t8SsEKfOt41JBob3t9Ien606ucEZmcgmpf5H0lzlL6oZS4vLpZERLur9y2YRhqd9AYPyhbrmUkrYmKAssv6PSCysf6z3-myn1YKr62Nxqfs7t1lZGZnZuH_I2j8LHl5PnVXWJxyADCY Oct 30 01:08:32.416116 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Domain-Id: None Oct 30 01:08:32.416116 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Domain-Name: None Oct 30 01:08:32.416116 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Identity-Status: Confirmed Oct 30 01:08:32.416116 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Is-Admin-Project: True Oct 30 01:08:32.416116 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Domain-Id: default Oct 30 01:08:32.416116 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Domain-Name: Default Oct 30 01:08:32.416116 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Id: 6b68b886c19c45d585409651f33b8626 Oct 30 01:08:32.416116 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Name: tempest-AttachVolumeNegativeTest-618009204 Oct 30 01:08:32.416116 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Role: reader,member Oct 30 01:08:32.416116 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Roles: reader,member Oct 30 01:08:32.417925 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_6b68b886c19c45d585409651f33b8626"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6b68b886c19c45d585409651f33b8626"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/6b68b886c19c45d585409651f33b8626"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6b68b886c19c45d585409651f33b8626"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 01:08:32.417925 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant: tempest-AttachVolumeNegativeTest-618009204 Oct 30 01:08:32.417925 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant-Id: 6b68b886c19c45d585409651f33b8626 Oct 30 01:08:32.417925 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-618009204 Oct 30 01:08:32.417925 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User: tempest-AttachVolumeNegativeTest-618009204-project Oct 30 01:08:32.417925 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Domain-Id: default Oct 30 01:08:32.417925 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Domain-Name: Default Oct 30 01:08:32.417925 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Id: ecaf8ec105c04b919256f852da859eba Oct 30 01:08:32.417925 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Name: tempest-AttachVolumeNegativeTest-618009204-project Oct 30 01:08:32.417925 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: method: > Oct 30 01:08:32.417925 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: exception: Instance 5632075c-7062-434e-b00c-bbf04c5ac04a could not be found. Oct 30 01:08:32.417925 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: action_args: {'id': '5632075c-7062-434e-b00c-bbf04c5ac04a'} Oct 30 01:08:32.418864 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 01:08:32.418864 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:32.418864 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 30 01:08:32.418864 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 30 01:08:32.418864 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 30 01:08:32.418864 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 30 01:08:32.418864 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 30 01:08:32.418864 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 30 01:08:32.418864 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 30 01:08:32.418864 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 30 01:08:32.418864 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5632075c-7062-434e-b00c-bbf04c5ac04a could not be found. Oct 30 01:08:32.418864 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:08:32.418864 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:08:32.418864 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:08:32.418864 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:32.418864 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:08:32.418864 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:08:32.418864 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 01:08:32.418864 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 01:08:32.418864 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 01:08:32.420364 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 01:08:32.420364 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 30 01:08:32.420364 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 30 01:08:32.420364 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 30 01:08:32.420364 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 30 01:08:32.420364 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 30 01:08:32.420364 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 30 01:08:32.420364 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 30 01:08:32.420364 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 30 01:08:32.420364 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5632075c-7062-434e-b00c-bbf04c5ac04a could not be found. Oct 30 01:08:32.420364 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:08:32.420364 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:08:32.420364 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:08:32.420364 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:32.420364 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:08:32.420364 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:08:32.420364 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 01:08:32.420364 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 01:08:32.420364 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5632075c-7062-434e-b00c-bbf04c5ac04a could not be found. Oct 30 01:08:32.420364 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:08:32.421897 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-9f8088e6-ea24-460b-9ae3-d2db847464e1 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Returning 404 to user: Instance 5632075c-7062-434e-b00c-bbf04c5ac04a could not be found. {{(pid=118961) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 01:08:32.421897 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-9f8088e6-ea24-460b-9ae3-d2db847464e1 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "DELETE /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a" status: 404 len: 111 microversion: 2.1 time: 0.030391 Oct 30 01:08:32.421897 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 641/1257] 10.222.0.16 () {58 vars in 1239 bytes} [Sun Oct 30 01:08:32 2022] DELETE /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 30 01:08:32.426082 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-fe0226f1-5dc2-4e3d-aefd-cd452369ca25 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:08:32.426699 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-fe0226f1-5dc2-4e3d-aefd-cd452369ca25 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5632075c-7062-434e-b00c-bbf04c5ac04a] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:08:32.432994 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-fe0226f1-5dc2-4e3d-aefd-cd452369ca25 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:32.433622 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-fe0226f1-5dc2-4e3d-aefd-cd452369ca25 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:32.434117 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-fe0226f1-5dc2-4e3d-aefd-cd452369ca25 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:32.434906 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-fe0226f1-5dc2-4e3d-aefd-cd452369ca25 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:32.435354 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-fe0226f1-5dc2-4e3d-aefd-cd452369ca25 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:32.435803 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-fe0226f1-5dc2-4e3d-aefd-cd452369ca25 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:32.449539 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.wsgi [None req-fe0226f1-5dc2-4e3d-aefd-cd452369ca25 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] HTTP exception thrown: Instance 5632075c-7062-434e-b00c-bbf04c5ac04a could not be found. Oct 30 01:08:32.451096 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-fe0226f1-5dc2-4e3d-aefd-cd452369ca25 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Request method failure captured: Oct 30 01:08:32.451096 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: request: GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a HTTP/1.1 Oct 30 01:08:32.451096 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept: application/json Oct 30 01:08:32.451096 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept-Encoding: identity Oct 30 01:08:32.451096 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Connection: close Oct 30 01:08:32.451096 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Length: 0 Oct 30 01:08:32.451096 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Type: application/json Oct 30 01:08:32.451096 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Host: 10.222.0.16 Oct 30 01:08:32.451096 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Openstack-System-Scope: None Oct 30 01:08:32.451096 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: User-Agent: python-urllib3/1.26.12 Oct 30 01:08:32.451096 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Auth-Token: gAAAAABjXcuedsaRjtnPRcHo4HqTnifOYe1vcHHjSzl7t8SsEKfOt41JBob3t9Ien606ucEZmcgmpf5H0lzlL6oZS4vLpZERLur9y2YRhqd9AYPyhbrmUkrYmKAssv6PSCysf6z3-myn1YKr62Nxqfs7t1lZGZnZuH_I2j8LHl5PnVXWJxyADCY Oct 30 01:08:32.451096 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Id: None Oct 30 01:08:32.451096 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Name: None Oct 30 01:08:32.451096 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Identity-Status: Confirmed Oct 30 01:08:32.451096 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Is-Admin-Project: True Oct 30 01:08:32.451096 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Id: default Oct 30 01:08:32.451096 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Name: Default Oct 30 01:08:32.451096 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Id: 6b68b886c19c45d585409651f33b8626 Oct 30 01:08:32.451096 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Name: tempest-AttachVolumeNegativeTest-618009204 Oct 30 01:08:32.451096 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Role: reader,member Oct 30 01:08:32.451096 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Roles: reader,member Oct 30 01:08:32.452690 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_6b68b886c19c45d585409651f33b8626"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6b68b886c19c45d585409651f33b8626"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/6b68b886c19c45d585409651f33b8626"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6b68b886c19c45d585409651f33b8626"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 01:08:32.452690 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant: tempest-AttachVolumeNegativeTest-618009204 Oct 30 01:08:32.452690 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Id: 6b68b886c19c45d585409651f33b8626 Oct 30 01:08:32.452690 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-618009204 Oct 30 01:08:32.452690 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User: tempest-AttachVolumeNegativeTest-618009204-project Oct 30 01:08:32.452690 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Id: default Oct 30 01:08:32.452690 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Name: Default Oct 30 01:08:32.452690 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Id: ecaf8ec105c04b919256f852da859eba Oct 30 01:08:32.452690 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Name: tempest-AttachVolumeNegativeTest-618009204-project Oct 30 01:08:32.452690 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: method: > Oct 30 01:08:32.452690 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: exception: Instance 5632075c-7062-434e-b00c-bbf04c5ac04a could not be found. Oct 30 01:08:32.452690 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: action_args: {'id': '5632075c-7062-434e-b00c-bbf04c5ac04a'} Oct 30 01:08:32.453531 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 01:08:32.453531 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:32.453531 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 30 01:08:32.453531 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 30 01:08:32.453531 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 30 01:08:32.453531 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 30 01:08:32.453531 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 30 01:08:32.453531 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 30 01:08:32.453531 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 30 01:08:32.453531 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 30 01:08:32.453531 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5632075c-7062-434e-b00c-bbf04c5ac04a could not be found. Oct 30 01:08:32.453531 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:08:32.453531 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:08:32.453531 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:08:32.453531 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:32.453531 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:08:32.453531 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:08:32.453531 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 01:08:32.453531 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 01:08:32.453531 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 01:08:32.455005 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 01:08:32.455005 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 30 01:08:32.455005 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 30 01:08:32.455005 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 30 01:08:32.455005 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 30 01:08:32.455005 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 30 01:08:32.455005 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 30 01:08:32.455005 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5632075c-7062-434e-b00c-bbf04c5ac04a could not be found. Oct 30 01:08:32.455005 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:08:32.455005 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:08:32.455005 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:08:32.455005 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:32.455005 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:08:32.455005 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:08:32.455005 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 01:08:32.455005 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 01:08:32.455005 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5632075c-7062-434e-b00c-bbf04c5ac04a could not be found. Oct 30 01:08:32.455005 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:08:32.456553 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-fe0226f1-5dc2-4e3d-aefd-cd452369ca25 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Returning 404 to user: Instance 5632075c-7062-434e-b00c-bbf04c5ac04a could not be found. {{(pid=118962) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 01:08:32.456553 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-fe0226f1-5dc2-4e3d-aefd-cd452369ca25 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a" status: 404 len: 111 microversion: 2.1 time: 0.030290 Oct 30 01:08:32.456553 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 617/1258] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:08:32 2022] GET /compute/v2.1/servers/5632075c-7062-434e-b00c-bbf04c5ac04a => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 30 01:08:33.516062 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-5da62e2f-8cce-4231-8764-5b5c1bc54d78 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:08:33.518267 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-5da62e2f-8cce-4231-8764-5b5c1bc54d78 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:08:33.525123 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5da62e2f-8cce-4231-8764-5b5c1bc54d78 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:33.525617 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5da62e2f-8cce-4231-8764-5b5c1bc54d78 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:33.526053 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5da62e2f-8cce-4231-8764-5b5c1bc54d78 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:33.526756 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5da62e2f-8cce-4231-8764-5b5c1bc54d78 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:33.527210 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5da62e2f-8cce-4231-8764-5b5c1bc54d78 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:33.527637 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5da62e2f-8cce-4231-8764-5b5c1bc54d78 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:33.559086 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-5da62e2f-8cce-4231-8764-5b5c1bc54d78 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.047086 Oct 30 01:08:33.559644 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 642/1259] 10.222.0.16 () {60 vars in 1343 bytes} [Sun Oct 30 01:08:33 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20/os-volume_attachments => generated 25 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:08:33.601140 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-d5b6beda-1ae3-4e83-9c06-67e110933748 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:08:33.601633 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-d5b6beda-1ae3-4e83-9c06-67e110933748 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5652be64-c2a3-4b24-a3aa-c962f204c06e] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:08:33.603999 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_api:SELECT=33,nova_api:DELETE=1 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:08:33.609481 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d5b6beda-1ae3-4e83-9c06-67e110933748 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:33.609960 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d5b6beda-1ae3-4e83-9c06-67e110933748 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:33.610629 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d5b6beda-1ae3-4e83-9c06-67e110933748 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:33.611349 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d5b6beda-1ae3-4e83-9c06-67e110933748 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:33.611802 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d5b6beda-1ae3-4e83-9c06-67e110933748 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:33.612218 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d5b6beda-1ae3-4e83-9c06-67e110933748 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:33.626708 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.wsgi [None req-d5b6beda-1ae3-4e83-9c06-67e110933748 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] HTTP exception thrown: Instance 5652be64-c2a3-4b24-a3aa-c962f204c06e could not be found. Oct 30 01:08:33.628291 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-d5b6beda-1ae3-4e83-9c06-67e110933748 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Request method failure captured: Oct 30 01:08:33.628291 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: request: DELETE /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e HTTP/1.1 Oct 30 01:08:33.628291 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept: application/json Oct 30 01:08:33.628291 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept-Encoding: identity Oct 30 01:08:33.628291 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Connection: close Oct 30 01:08:33.628291 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Length: 0 Oct 30 01:08:33.628291 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Type: application/json Oct 30 01:08:33.628291 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Host: 10.222.0.16 Oct 30 01:08:33.628291 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Openstack-System-Scope: None Oct 30 01:08:33.628291 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: User-Agent: python-urllib3/1.26.12 Oct 30 01:08:33.628291 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Auth-Token: gAAAAABjXcuedsaRjtnPRcHo4HqTnifOYe1vcHHjSzl7t8SsEKfOt41JBob3t9Ien606ucEZmcgmpf5H0lzlL6oZS4vLpZERLur9y2YRhqd9AYPyhbrmUkrYmKAssv6PSCysf6z3-myn1YKr62Nxqfs7t1lZGZnZuH_I2j8LHl5PnVXWJxyADCY Oct 30 01:08:33.628291 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Id: None Oct 30 01:08:33.628291 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Name: None Oct 30 01:08:33.628291 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Identity-Status: Confirmed Oct 30 01:08:33.628291 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Is-Admin-Project: True Oct 30 01:08:33.628291 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Id: default Oct 30 01:08:33.628291 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Name: Default Oct 30 01:08:33.628291 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Id: 6b68b886c19c45d585409651f33b8626 Oct 30 01:08:33.628291 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Name: tempest-AttachVolumeNegativeTest-618009204 Oct 30 01:08:33.628291 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Role: reader,member Oct 30 01:08:33.628291 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Roles: reader,member Oct 30 01:08:33.629988 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_6b68b886c19c45d585409651f33b8626"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6b68b886c19c45d585409651f33b8626"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/6b68b886c19c45d585409651f33b8626"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6b68b886c19c45d585409651f33b8626"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 01:08:33.629988 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant: tempest-AttachVolumeNegativeTest-618009204 Oct 30 01:08:33.629988 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Id: 6b68b886c19c45d585409651f33b8626 Oct 30 01:08:33.629988 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-618009204 Oct 30 01:08:33.629988 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User: tempest-AttachVolumeNegativeTest-618009204-project Oct 30 01:08:33.629988 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Id: default Oct 30 01:08:33.629988 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Name: Default Oct 30 01:08:33.629988 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Id: ecaf8ec105c04b919256f852da859eba Oct 30 01:08:33.629988 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Name: tempest-AttachVolumeNegativeTest-618009204-project Oct 30 01:08:33.629988 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: method: > Oct 30 01:08:33.629988 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: exception: Instance 5652be64-c2a3-4b24-a3aa-c962f204c06e could not be found. Oct 30 01:08:33.629988 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: action_args: {'id': '5652be64-c2a3-4b24-a3aa-c962f204c06e'} Oct 30 01:08:33.630780 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 01:08:33.630780 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:33.630780 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 30 01:08:33.630780 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 30 01:08:33.630780 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 30 01:08:33.630780 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 30 01:08:33.630780 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 30 01:08:33.630780 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 30 01:08:33.630780 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 30 01:08:33.630780 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 30 01:08:33.630780 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5652be64-c2a3-4b24-a3aa-c962f204c06e could not be found. Oct 30 01:08:33.630780 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:08:33.630780 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:08:33.630780 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:08:33.630780 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:33.630780 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:08:33.630780 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:08:33.630780 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 01:08:33.630780 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 01:08:33.630780 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 01:08:33.632090 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 01:08:33.632090 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 30 01:08:33.632090 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 30 01:08:33.632090 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 30 01:08:33.632090 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 30 01:08:33.632090 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 30 01:08:33.632090 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 30 01:08:33.632090 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 30 01:08:33.632090 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 30 01:08:33.632090 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5652be64-c2a3-4b24-a3aa-c962f204c06e could not be found. Oct 30 01:08:33.632090 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:08:33.632090 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:08:33.632090 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:08:33.632090 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:33.632090 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:08:33.632090 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:08:33.632090 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 01:08:33.632090 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 01:08:33.632090 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5652be64-c2a3-4b24-a3aa-c962f204c06e could not be found. Oct 30 01:08:33.632090 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:08:33.633383 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-d5b6beda-1ae3-4e83-9c06-67e110933748 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Returning 404 to user: Instance 5652be64-c2a3-4b24-a3aa-c962f204c06e could not be found. {{(pid=118962) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 01:08:33.633487 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-b535a656-3394-4922-aeb2-2ff1f9219a5b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:08:33.633586 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-d5b6beda-1ae3-4e83-9c06-67e110933748 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "DELETE /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e" status: 404 len: 111 microversion: 2.1 time: 0.032373 Oct 30 01:08:33.633675 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-b535a656-3394-4922-aeb2-2ff1f9219a5b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:08:33.633756 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 618/1260] 10.222.0.16 () {58 vars in 1239 bytes} [Sun Oct 30 01:08:33 2022] DELETE /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 30 01:08:33.635919 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b535a656-3394-4922-aeb2-2ff1f9219a5b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:33.636409 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b535a656-3394-4922-aeb2-2ff1f9219a5b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:33.636835 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b535a656-3394-4922-aeb2-2ff1f9219a5b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:33.637522 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b535a656-3394-4922-aeb2-2ff1f9219a5b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:33.638008 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-d10f5c20-5bed-4831-85e4-1876305af040 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:08:33.638215 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b535a656-3394-4922-aeb2-2ff1f9219a5b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:33.638636 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-d10f5c20-5bed-4831-85e4-1876305af040 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] [instance: 5652be64-c2a3-4b24-a3aa-c962f204c06e] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:08:33.638795 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b535a656-3394-4922-aeb2-2ff1f9219a5b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:33.644607 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d10f5c20-5bed-4831-85e4-1876305af040 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:33.645210 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d10f5c20-5bed-4831-85e4-1876305af040 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:33.645641 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d10f5c20-5bed-4831-85e4-1876305af040 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:33.646322 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d10f5c20-5bed-4831-85e4-1876305af040 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:33.646761 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d10f5c20-5bed-4831-85e4-1876305af040 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:33.647168 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d10f5c20-5bed-4831-85e4-1876305af040 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:33.660629 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.wsgi [None req-d10f5c20-5bed-4831-85e4-1876305af040 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] HTTP exception thrown: Instance 5652be64-c2a3-4b24-a3aa-c962f204c06e could not be found. Oct 30 01:08:33.662462 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-d10f5c20-5bed-4831-85e4-1876305af040 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Request method failure captured: Oct 30 01:08:33.662462 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: request: GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e HTTP/1.1 Oct 30 01:08:33.662462 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept: application/json Oct 30 01:08:33.662462 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept-Encoding: identity Oct 30 01:08:33.662462 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Connection: close Oct 30 01:08:33.662462 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Length: 0 Oct 30 01:08:33.662462 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Type: application/json Oct 30 01:08:33.662462 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Host: 10.222.0.16 Oct 30 01:08:33.662462 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Openstack-System-Scope: None Oct 30 01:08:33.662462 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: User-Agent: python-urllib3/1.26.12 Oct 30 01:08:33.662462 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Auth-Token: gAAAAABjXcuedsaRjtnPRcHo4HqTnifOYe1vcHHjSzl7t8SsEKfOt41JBob3t9Ien606ucEZmcgmpf5H0lzlL6oZS4vLpZERLur9y2YRhqd9AYPyhbrmUkrYmKAssv6PSCysf6z3-myn1YKr62Nxqfs7t1lZGZnZuH_I2j8LHl5PnVXWJxyADCY Oct 30 01:08:33.662462 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Id: None Oct 30 01:08:33.662462 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Name: None Oct 30 01:08:33.662462 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Identity-Status: Confirmed Oct 30 01:08:33.662462 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Is-Admin-Project: True Oct 30 01:08:33.662462 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Id: default Oct 30 01:08:33.662462 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Name: Default Oct 30 01:08:33.662462 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Id: 6b68b886c19c45d585409651f33b8626 Oct 30 01:08:33.662462 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Name: tempest-AttachVolumeNegativeTest-618009204 Oct 30 01:08:33.662462 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Role: reader,member Oct 30 01:08:33.662462 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Roles: reader,member Oct 30 01:08:33.664737 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_6b68b886c19c45d585409651f33b8626"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6b68b886c19c45d585409651f33b8626"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/6b68b886c19c45d585409651f33b8626"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6b68b886c19c45d585409651f33b8626"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 01:08:33.664737 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant: tempest-AttachVolumeNegativeTest-618009204 Oct 30 01:08:33.664737 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Id: 6b68b886c19c45d585409651f33b8626 Oct 30 01:08:33.664737 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-618009204 Oct 30 01:08:33.664737 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User: tempest-AttachVolumeNegativeTest-618009204-project Oct 30 01:08:33.664737 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Id: default Oct 30 01:08:33.664737 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Name: Default Oct 30 01:08:33.664737 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Id: ecaf8ec105c04b919256f852da859eba Oct 30 01:08:33.664737 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Name: tempest-AttachVolumeNegativeTest-618009204-project Oct 30 01:08:33.664737 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: method: > Oct 30 01:08:33.664737 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: exception: Instance 5652be64-c2a3-4b24-a3aa-c962f204c06e could not be found. Oct 30 01:08:33.664737 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: action_args: {'id': '5652be64-c2a3-4b24-a3aa-c962f204c06e'} Oct 30 01:08:33.665950 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 01:08:33.665950 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:33.665950 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 30 01:08:33.665950 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 30 01:08:33.665950 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 30 01:08:33.665950 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 30 01:08:33.665950 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 30 01:08:33.665950 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 30 01:08:33.665950 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 30 01:08:33.665950 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 30 01:08:33.665950 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5652be64-c2a3-4b24-a3aa-c962f204c06e could not be found. Oct 30 01:08:33.665950 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:08:33.665950 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:08:33.665950 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:08:33.665950 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:33.665950 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:08:33.665950 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:08:33.665950 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 01:08:33.665950 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 01:08:33.665950 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 01:08:33.667548 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 01:08:33.667548 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 30 01:08:33.667548 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 30 01:08:33.667548 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 30 01:08:33.667548 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 30 01:08:33.667548 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 30 01:08:33.667548 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 30 01:08:33.667548 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5652be64-c2a3-4b24-a3aa-c962f204c06e could not be found. Oct 30 01:08:33.667548 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:08:33.667548 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:08:33.667548 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:08:33.667548 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:33.667548 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:08:33.667548 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:08:33.667548 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 01:08:33.667548 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 01:08:33.667548 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5652be64-c2a3-4b24-a3aa-c962f204c06e could not be found. Oct 30 01:08:33.667548 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:08:33.670109 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-d10f5c20-5bed-4831-85e4-1876305af040 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] Returning 404 to user: Instance 5652be64-c2a3-4b24-a3aa-c962f204c06e could not be found. {{(pid=118962) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 01:08:33.670777 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-b535a656-3394-4922-aeb2-2ff1f9219a5b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Going to try to terminate instance {{(pid=118961) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 30 01:08:33.670934 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-d10f5c20-5bed-4831-85e4-1876305af040 tempest-AttachVolumeNegativeTest-618009204 tempest-AttachVolumeNegativeTest-618009204-project] 10.222.0.16 "GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e" status: 404 len: 111 microversion: 2.1 time: 0.029565 Oct 30 01:08:33.670934 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 619/1261] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:08:33 2022] GET /compute/v2.1/servers/5652be64-c2a3-4b24-a3aa-c962f204c06e => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 30 01:08:33.747944 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-b535a656-3394-4922-aeb2-2ff1f9219a5b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "DELETE /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 204 len: 0 microversion: 2.20 time: 0.122052 Oct 30 01:08:33.748836 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 643/1262] 10.222.0.16 () {60 vars in 1280 bytes} [Sun Oct 30 01:08:33 2022] DELETE /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 30 01:08:33.759325 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-d59ea825-de16-4e37-a7f6-ee4ade580864 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:08:33.760301 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-d59ea825-de16-4e37-a7f6-ee4ade580864 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:08:33.766720 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d59ea825-de16-4e37-a7f6-ee4ade580864 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:33.767190 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d59ea825-de16-4e37-a7f6-ee4ade580864 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:33.767610 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d59ea825-de16-4e37-a7f6-ee4ade580864 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:33.768314 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d59ea825-de16-4e37-a7f6-ee4ade580864 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:33.768761 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d59ea825-de16-4e37-a7f6-ee4ade580864 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:33.769171 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d59ea825-de16-4e37-a7f6-ee4ade580864 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:33.982395 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-d59ea825-de16-4e37-a7f6-ee4ade580864 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:08:33.990858 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-d59ea825-de16-4e37-a7f6-ee4ade580864 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:08:33.992332 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-d59ea825-de16-4e37-a7f6-ee4ade580864 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:08:33.993500 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-d59ea825-de16-4e37-a7f6-ee4ade580864 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1723 microversion: 2.20 time: 0.239573 Oct 30 01:08:33.994060 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 620/1263] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:08:33 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1723 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:08:35.020940 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-2c4aa7af-c878-4db0-8cd9-5ec4d5a063ba tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:08:35.022213 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-2c4aa7af-c878-4db0-8cd9-5ec4d5a063ba tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:08:35.029433 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2c4aa7af-c878-4db0-8cd9-5ec4d5a063ba tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:35.029906 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2c4aa7af-c878-4db0-8cd9-5ec4d5a063ba tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:35.030345 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2c4aa7af-c878-4db0-8cd9-5ec4d5a063ba tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:35.031047 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2c4aa7af-c878-4db0-8cd9-5ec4d5a063ba tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:35.031486 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2c4aa7af-c878-4db0-8cd9-5ec4d5a063ba tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:35.031922 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2c4aa7af-c878-4db0-8cd9-5ec4d5a063ba tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:35.245398 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-2c4aa7af-c878-4db0-8cd9-5ec4d5a063ba tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:08:35.254235 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-2c4aa7af-c878-4db0-8cd9-5ec4d5a063ba tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:08:35.256494 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-2c4aa7af-c878-4db0-8cd9-5ec4d5a063ba tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:08:35.257786 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-2c4aa7af-c878-4db0-8cd9-5ec4d5a063ba tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1723 microversion: 2.20 time: 0.241261 Oct 30 01:08:35.259021 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 644/1264] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:08:35 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1723 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:08:36.286716 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-ba5203e2-513e-4e97-983e-994a3563a7ce tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:08:36.287398 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-ba5203e2-513e-4e97-983e-994a3563a7ce tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:08:36.294231 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ba5203e2-513e-4e97-983e-994a3563a7ce tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:36.294804 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ba5203e2-513e-4e97-983e-994a3563a7ce tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:36.295297 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ba5203e2-513e-4e97-983e-994a3563a7ce tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:36.296138 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ba5203e2-513e-4e97-983e-994a3563a7ce tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:36.296634 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ba5203e2-513e-4e97-983e-994a3563a7ce tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:36.297148 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ba5203e2-513e-4e97-983e-994a3563a7ce tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:36.461809 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-ba5203e2-513e-4e97-983e-994a3563a7ce tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:08:36.472765 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-ba5203e2-513e-4e97-983e-994a3563a7ce tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:08:36.475363 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-ba5203e2-513e-4e97-983e-994a3563a7ce tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:08:36.477112 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-ba5203e2-513e-4e97-983e-994a3563a7ce tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1723 microversion: 2.20 time: 0.194153 Oct 30 01:08:36.478564 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 621/1265] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:08:36 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1723 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:08:37.504022 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-836a4701-f9b8-466b-96b1-558681a4b690 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:08:37.504623 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-836a4701-f9b8-466b-96b1-558681a4b690 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:08:37.511224 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-836a4701-f9b8-466b-96b1-558681a4b690 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:37.511681 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-836a4701-f9b8-466b-96b1-558681a4b690 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:37.512441 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-836a4701-f9b8-466b-96b1-558681a4b690 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:37.513173 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-836a4701-f9b8-466b-96b1-558681a4b690 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:37.513625 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-836a4701-f9b8-466b-96b1-558681a4b690 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:37.514051 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-836a4701-f9b8-466b-96b1-558681a4b690 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:37.647195 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-836a4701-f9b8-466b-96b1-558681a4b690 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:08:37.655802 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-836a4701-f9b8-466b-96b1-558681a4b690 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:08:37.657280 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-836a4701-f9b8-466b-96b1-558681a4b690 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:08:37.658432 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-836a4701-f9b8-466b-96b1-558681a4b690 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1723 microversion: 2.20 time: 0.158637 Oct 30 01:08:37.658988 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 645/1266] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:08:37 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1723 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:08:38.134500 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-5eca06bc-711d-4ece-8506-b3ad196d18e9 tempest-VolumesNegativeTest-1555677896 tempest-VolumesNegativeTest-1555677896-project] Action: 'create', calling method: .version_select at 0x7f0eb0947550>, body: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-86696172", "metadata": {"Type": "work"}}} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:08:38.156189 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-5eca06bc-711d-4ece-8506-b3ad196d18e9 tempest-VolumesNegativeTest-1555677896 tempest-VolumesNegativeTest-1555677896-project] Request method failure captured: Oct 30 01:08:38.156189 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Oct 30 01:08:38.156189 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept: application/json Oct 30 01:08:38.156189 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept-Encoding: identity Oct 30 01:08:38.156189 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Connection: close Oct 30 01:08:38.156189 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Length: 120 Oct 30 01:08:38.156189 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Type: application/json Oct 30 01:08:38.156189 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Host: 10.222.0.16 Oct 30 01:08:38.156189 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Openstack-System-Scope: None Oct 30 01:08:38.156189 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: User-Agent: python-urllib3/1.26.12 Oct 30 01:08:38.156189 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Auth-Token: gAAAAABjXc6VrQlE6SHfwoCxrZZqW7NGWSuH_3D_l7GnPJbcH3QXg7CchMG0txgs0Z2hhFPPMw5BhOiE2zzvqaev72qVL5vpi9jQYpYsPSu7ADgn8uTYJHpbz9RJJA-ozO7MnszbCv5ba9nxzoD-BGIwEbqVPQqZWCjwAF8YAhZTWQgF4kZP9Rc Oct 30 01:08:38.156189 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Id: None Oct 30 01:08:38.156189 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Name: None Oct 30 01:08:38.156189 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Identity-Status: Confirmed Oct 30 01:08:38.156189 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Is-Admin-Project: True Oct 30 01:08:38.156189 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Id: default Oct 30 01:08:38.156189 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Name: Default Oct 30 01:08:38.156189 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Id: 4cb13eb3ad574ce9a9150be4773c00fa Oct 30 01:08:38.156189 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Name: tempest-VolumesNegativeTest-1555677896 Oct 30 01:08:38.156189 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Role: reader,member Oct 30 01:08:38.156189 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Roles: reader,member Oct 30 01:08:38.157999 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_4cb13eb3ad574ce9a9150be4773c00fa"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/4cb13eb3ad574ce9a9150be4773c00fa"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/4cb13eb3ad574ce9a9150be4773c00fa"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/4cb13eb3ad574ce9a9150be4773c00fa"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 01:08:38.157999 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant: tempest-VolumesNegativeTest-1555677896 Oct 30 01:08:38.157999 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Id: 4cb13eb3ad574ce9a9150be4773c00fa Oct 30 01:08:38.157999 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Name: tempest-VolumesNegativeTest-1555677896 Oct 30 01:08:38.157999 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User: tempest-VolumesNegativeTest-1555677896-project Oct 30 01:08:38.157999 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Id: default Oct 30 01:08:38.157999 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Name: Default Oct 30 01:08:38.157999 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Id: 25c6ea71c72747c59d16087ace588111 Oct 30 01:08:38.157999 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Name: tempest-VolumesNegativeTest-1555677896-project Oct 30 01:08:38.157999 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Oct 30 01:08:38.157999 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-86696172", "metadata": {"Type": "work"}}} Oct 30 01:08:38.157999 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: method: Oct 30 01:08:38.157999 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: exception: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Oct 30 01:08:38.159045 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: action_args: {'body': {'volume': {'size': '#$%', 'display_name': 'tempest-VolumesNegativeTest-Volume-86696172', 'metadata': {'Type': 'work'}}}} Oct 30 01:08:38.159045 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 01:08:38.159045 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:38.159045 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 30 01:08:38.159045 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 30 01:08:38.159045 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 30 01:08:38.159045 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise error Oct 30 01:08:38.159045 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '#$%' does not match '^[0-9]+$' Oct 30 01:08:38.159045 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:08:38.159045 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Oct 30 01:08:38.159045 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Oct 30 01:08:38.159045 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:08:38.159045 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi On instance['volume']: Oct 30 01:08:38.159045 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi '#$%' Oct 30 01:08:38.159045 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:08:38.159045 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:08:38.159045 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:08:38.159045 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:38.159045 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:08:38.159045 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:08:38.159045 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 01:08:38.159045 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 01:08:38.160755 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 30 01:08:38.160755 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 30 01:08:38.160755 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 01:08:38.160755 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 01:08:38.160755 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 30 01:08:38.160755 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 30 01:08:38.160755 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 30 01:08:38.160755 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 30 01:08:38.160755 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 30 01:08:38.160755 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 30 01:08:38.160755 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Oct 30 01:08:38.160755 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:08:38.160755 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:08:38.160755 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:08:38.160755 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:38.160755 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:08:38.160755 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:08:38.160755 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 30 01:08:38.160755 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 30 01:08:38.162431 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Oct 30 01:08:38.162431 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:08:38.162431 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-5eca06bc-711d-4ece-8506-b3ad196d18e9 tempest-VolumesNegativeTest-1555677896 tempest-VolumesNegativeTest-1555677896-project] Returning 400 to user: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' {{(pid=118962) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 01:08:38.162431 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-5eca06bc-711d-4ece-8506-b3ad196d18e9 tempest-VolumesNegativeTest-1555677896 tempest-VolumesNegativeTest-1555677896-project] 10.222.0.16 "POST /compute/v2.1/os-volumes" status: 400 len: 127 microversion: 2.1 time: 0.147351 Oct 30 01:08:38.162431 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 622/1267] 10.222.0.16 () {60 vars in 1156 bytes} [Sun Oct 30 01:08:38 2022] POST /compute/v2.1/os-volumes => generated 127 bytes in 149 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 30 01:08:38.168062 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-5234002d-3723-4629-82e6-20a7e6e12664 tempest-VolumesNegativeTest-1555677896 tempest-VolumesNegativeTest-1555677896-project] Action: 'create', calling method: .version_select at 0x7f0eaba22b80>, body: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-1081922299", "metadata": {"Type": "work"}}} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:08:38.173009 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-5234002d-3723-4629-82e6-20a7e6e12664 tempest-VolumesNegativeTest-1555677896 tempest-VolumesNegativeTest-1555677896-project] Request method failure captured: Oct 30 01:08:38.173009 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Oct 30 01:08:38.173009 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Accept: application/json Oct 30 01:08:38.173009 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Accept-Encoding: identity Oct 30 01:08:38.173009 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Connection: close Oct 30 01:08:38.173009 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Content-Length: 120 Oct 30 01:08:38.173009 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Content-Type: application/json Oct 30 01:08:38.173009 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Host: 10.222.0.16 Oct 30 01:08:38.173009 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Openstack-System-Scope: None Oct 30 01:08:38.173009 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: User-Agent: python-urllib3/1.26.12 Oct 30 01:08:38.173009 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Auth-Token: gAAAAABjXc6VrQlE6SHfwoCxrZZqW7NGWSuH_3D_l7GnPJbcH3QXg7CchMG0txgs0Z2hhFPPMw5BhOiE2zzvqaev72qVL5vpi9jQYpYsPSu7ADgn8uTYJHpbz9RJJA-ozO7MnszbCv5ba9nxzoD-BGIwEbqVPQqZWCjwAF8YAhZTWQgF4kZP9Rc Oct 30 01:08:38.173009 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Domain-Id: None Oct 30 01:08:38.173009 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Domain-Name: None Oct 30 01:08:38.173009 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Identity-Status: Confirmed Oct 30 01:08:38.173009 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Is-Admin-Project: True Oct 30 01:08:38.173009 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Domain-Id: default Oct 30 01:08:38.173009 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Domain-Name: Default Oct 30 01:08:38.173009 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Id: 4cb13eb3ad574ce9a9150be4773c00fa Oct 30 01:08:38.173009 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Name: tempest-VolumesNegativeTest-1555677896 Oct 30 01:08:38.173009 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Role: reader,member Oct 30 01:08:38.173009 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Roles: reader,member Oct 30 01:08:38.174699 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_4cb13eb3ad574ce9a9150be4773c00fa"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/4cb13eb3ad574ce9a9150be4773c00fa"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/4cb13eb3ad574ce9a9150be4773c00fa"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/4cb13eb3ad574ce9a9150be4773c00fa"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 01:08:38.174699 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant: tempest-VolumesNegativeTest-1555677896 Oct 30 01:08:38.174699 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant-Id: 4cb13eb3ad574ce9a9150be4773c00fa Oct 30 01:08:38.174699 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant-Name: tempest-VolumesNegativeTest-1555677896 Oct 30 01:08:38.174699 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User: tempest-VolumesNegativeTest-1555677896-project Oct 30 01:08:38.174699 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Domain-Id: default Oct 30 01:08:38.174699 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Domain-Name: Default Oct 30 01:08:38.174699 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Id: 25c6ea71c72747c59d16087ace588111 Oct 30 01:08:38.174699 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Name: tempest-VolumesNegativeTest-1555677896-project Oct 30 01:08:38.174699 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Oct 30 01:08:38.174699 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-1081922299", "metadata": {"Type": "work"}}} Oct 30 01:08:38.174699 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: method: Oct 30 01:08:38.174699 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: exception: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Oct 30 01:08:38.175820 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: action_args: {'body': {'volume': {'size': '0', 'display_name': 'tempest-VolumesNegativeTest-Volume-1081922299', 'metadata': {'Type': 'work'}}}} Oct 30 01:08:38.175820 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 01:08:38.175820 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:38.175820 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 30 01:08:38.175820 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 30 01:08:38.175820 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 30 01:08:38.175820 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise error Oct 30 01:08:38.175820 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 0 is less than the minimum of 1 Oct 30 01:08:38.175820 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:08:38.175820 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Failed validating 'minimum' in schema['properties']['volume']['properties']['size']: Oct 30 01:08:38.175820 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Oct 30 01:08:38.175820 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:08:38.175820 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi On instance['volume']: Oct 30 01:08:38.175820 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi '0' Oct 30 01:08:38.175820 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:08:38.175820 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:08:38.175820 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:08:38.175820 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:38.175820 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:08:38.175820 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:08:38.175820 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 01:08:38.175820 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 01:08:38.177546 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 30 01:08:38.177546 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 30 01:08:38.177546 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 01:08:38.177546 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 01:08:38.177546 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 30 01:08:38.177546 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 30 01:08:38.177546 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 30 01:08:38.177546 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 30 01:08:38.177546 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 30 01:08:38.177546 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 30 01:08:38.177546 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Oct 30 01:08:38.177546 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:08:38.177546 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:08:38.177546 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:08:38.177546 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:38.177546 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:08:38.177546 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:08:38.177546 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 30 01:08:38.177546 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 30 01:08:38.178899 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Oct 30 01:08:38.178899 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:08:38.178899 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-5234002d-3723-4629-82e6-20a7e6e12664 tempest-VolumesNegativeTest-1555677896 tempest-VolumesNegativeTest-1555677896-project] Returning 400 to user: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 {{(pid=118961) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 01:08:38.178899 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-5234002d-3723-4629-82e6-20a7e6e12664 tempest-VolumesNegativeTest-1555677896 tempest-VolumesNegativeTest-1555677896-project] 10.222.0.16 "POST /compute/v2.1/os-volumes" status: 400 len: 125 microversion: 2.1 time: 0.010279 Oct 30 01:08:38.178899 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 646/1268] 10.222.0.16 () {60 vars in 1156 bytes} [Sun Oct 30 01:08:38 2022] POST /compute/v2.1/os-volumes => generated 125 bytes in 11 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 30 01:08:38.183676 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-438e5315-2009-440d-a747-d33c97f5af8c tempest-VolumesNegativeTest-1555677896 tempest-VolumesNegativeTest-1555677896-project] Action: 'create', calling method: .version_select at 0x7f0eafa1daf0>, body: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-104324607", "metadata": {"Type": "work"}}} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:08:38.187636 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-438e5315-2009-440d-a747-d33c97f5af8c tempest-VolumesNegativeTest-1555677896 tempest-VolumesNegativeTest-1555677896-project] Request method failure captured: Oct 30 01:08:38.187636 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Oct 30 01:08:38.187636 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept: application/json Oct 30 01:08:38.187636 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept-Encoding: identity Oct 30 01:08:38.187636 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Connection: close Oct 30 01:08:38.187636 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Length: 118 Oct 30 01:08:38.187636 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Type: application/json Oct 30 01:08:38.187636 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Host: 10.222.0.16 Oct 30 01:08:38.187636 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Openstack-System-Scope: None Oct 30 01:08:38.187636 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: User-Agent: python-urllib3/1.26.12 Oct 30 01:08:38.187636 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Auth-Token: gAAAAABjXc6VrQlE6SHfwoCxrZZqW7NGWSuH_3D_l7GnPJbcH3QXg7CchMG0txgs0Z2hhFPPMw5BhOiE2zzvqaev72qVL5vpi9jQYpYsPSu7ADgn8uTYJHpbz9RJJA-ozO7MnszbCv5ba9nxzoD-BGIwEbqVPQqZWCjwAF8YAhZTWQgF4kZP9Rc Oct 30 01:08:38.187636 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Id: None Oct 30 01:08:38.187636 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Name: None Oct 30 01:08:38.187636 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Identity-Status: Confirmed Oct 30 01:08:38.187636 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Is-Admin-Project: True Oct 30 01:08:38.187636 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Id: default Oct 30 01:08:38.187636 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Name: Default Oct 30 01:08:38.187636 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Id: 4cb13eb3ad574ce9a9150be4773c00fa Oct 30 01:08:38.187636 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Name: tempest-VolumesNegativeTest-1555677896 Oct 30 01:08:38.187636 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Role: reader,member Oct 30 01:08:38.187636 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Roles: reader,member Oct 30 01:08:38.189342 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_4cb13eb3ad574ce9a9150be4773c00fa"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/4cb13eb3ad574ce9a9150be4773c00fa"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/4cb13eb3ad574ce9a9150be4773c00fa"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/4cb13eb3ad574ce9a9150be4773c00fa"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 01:08:38.189342 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant: tempest-VolumesNegativeTest-1555677896 Oct 30 01:08:38.189342 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Id: 4cb13eb3ad574ce9a9150be4773c00fa Oct 30 01:08:38.189342 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Name: tempest-VolumesNegativeTest-1555677896 Oct 30 01:08:38.189342 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User: tempest-VolumesNegativeTest-1555677896-project Oct 30 01:08:38.189342 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Id: default Oct 30 01:08:38.189342 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Name: Default Oct 30 01:08:38.189342 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Id: 25c6ea71c72747c59d16087ace588111 Oct 30 01:08:38.189342 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Name: tempest-VolumesNegativeTest-1555677896-project Oct 30 01:08:38.189342 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Oct 30 01:08:38.189342 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-104324607", "metadata": {"Type": "work"}}} Oct 30 01:08:38.189342 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: method: Oct 30 01:08:38.189342 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: exception: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Oct 30 01:08:38.190375 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: action_args: {'body': {'volume': {'size': '', 'display_name': 'tempest-VolumesNegativeTest-Volume-104324607', 'metadata': {'Type': 'work'}}}} Oct 30 01:08:38.190375 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 01:08:38.190375 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:38.190375 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 30 01:08:38.190375 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 30 01:08:38.190375 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 30 01:08:38.190375 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise error Oct 30 01:08:38.190375 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' does not match '^[0-9]+$' Oct 30 01:08:38.190375 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:08:38.190375 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Oct 30 01:08:38.190375 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Oct 30 01:08:38.190375 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:08:38.190375 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi On instance['volume']: Oct 30 01:08:38.190375 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi '' Oct 30 01:08:38.190375 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:08:38.190375 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:08:38.190375 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:08:38.190375 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:38.190375 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:08:38.190375 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:08:38.190375 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 01:08:38.190375 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 01:08:38.191951 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 30 01:08:38.191951 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 30 01:08:38.191951 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 01:08:38.191951 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 01:08:38.191951 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 30 01:08:38.191951 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 30 01:08:38.191951 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 30 01:08:38.191951 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 30 01:08:38.191951 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 30 01:08:38.191951 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 30 01:08:38.191951 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Oct 30 01:08:38.191951 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:08:38.191951 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:08:38.191951 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:08:38.191951 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:38.191951 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:08:38.191951 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:08:38.191951 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 30 01:08:38.191951 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 30 01:08:38.193678 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Oct 30 01:08:38.193678 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:08:38.193678 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-438e5315-2009-440d-a747-d33c97f5af8c tempest-VolumesNegativeTest-1555677896 tempest-VolumesNegativeTest-1555677896-project] Returning 400 to user: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' {{(pid=118962) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 01:08:38.193678 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-438e5315-2009-440d-a747-d33c97f5af8c tempest-VolumesNegativeTest-1555677896 tempest-VolumesNegativeTest-1555677896-project] 10.222.0.16 "POST /compute/v2.1/os-volumes" status: 400 len: 121 microversion: 2.1 time: 0.009173 Oct 30 01:08:38.193678 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 623/1269] 10.222.0.16 () {60 vars in 1156 bytes} [Sun Oct 30 01:08:38 2022] POST /compute/v2.1/os-volumes => generated 121 bytes in 10 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 30 01:08:38.198286 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-51fa2491-40d4-49b9-91dd-019075010000 tempest-VolumesNegativeTest-1555677896 tempest-VolumesNegativeTest-1555677896-project] Calling method '.version_select at 0x7f0eab9e59d0>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:08:38.232606 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.wsgi [None req-51fa2491-40d4-49b9-91dd-019075010000 tempest-VolumesNegativeTest-1555677896 tempest-VolumesNegativeTest-1555677896-project] HTTP exception thrown: Volume tempest-invalid-1906016073 could not be found. Oct 30 01:08:38.235754 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-51fa2491-40d4-49b9-91dd-019075010000 tempest-VolumesNegativeTest-1555677896 tempest-VolumesNegativeTest-1555677896-project] Request method failure captured: Oct 30 01:08:38.235754 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: request: DELETE /compute/v2.1/os-volumes/tempest-invalid-1906016073 HTTP/1.1 Oct 30 01:08:38.235754 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Accept: application/json Oct 30 01:08:38.235754 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Accept-Encoding: identity Oct 30 01:08:38.235754 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Connection: close Oct 30 01:08:38.235754 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Content-Length: 0 Oct 30 01:08:38.235754 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Content-Type: application/json Oct 30 01:08:38.235754 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Host: 10.222.0.16 Oct 30 01:08:38.235754 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Openstack-System-Scope: None Oct 30 01:08:38.235754 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: User-Agent: python-urllib3/1.26.12 Oct 30 01:08:38.235754 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Auth-Token: gAAAAABjXc6VrQlE6SHfwoCxrZZqW7NGWSuH_3D_l7GnPJbcH3QXg7CchMG0txgs0Z2hhFPPMw5BhOiE2zzvqaev72qVL5vpi9jQYpYsPSu7ADgn8uTYJHpbz9RJJA-ozO7MnszbCv5ba9nxzoD-BGIwEbqVPQqZWCjwAF8YAhZTWQgF4kZP9Rc Oct 30 01:08:38.235754 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Domain-Id: None Oct 30 01:08:38.235754 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Domain-Name: None Oct 30 01:08:38.235754 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Identity-Status: Confirmed Oct 30 01:08:38.235754 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Is-Admin-Project: True Oct 30 01:08:38.235754 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Domain-Id: default Oct 30 01:08:38.235754 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Domain-Name: Default Oct 30 01:08:38.235754 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Id: 4cb13eb3ad574ce9a9150be4773c00fa Oct 30 01:08:38.235754 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Name: tempest-VolumesNegativeTest-1555677896 Oct 30 01:08:38.235754 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Role: reader,member Oct 30 01:08:38.235754 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Roles: reader,member Oct 30 01:08:38.237367 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_4cb13eb3ad574ce9a9150be4773c00fa"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/4cb13eb3ad574ce9a9150be4773c00fa"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/4cb13eb3ad574ce9a9150be4773c00fa"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/4cb13eb3ad574ce9a9150be4773c00fa"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 01:08:38.237367 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant: tempest-VolumesNegativeTest-1555677896 Oct 30 01:08:38.237367 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant-Id: 4cb13eb3ad574ce9a9150be4773c00fa Oct 30 01:08:38.237367 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant-Name: tempest-VolumesNegativeTest-1555677896 Oct 30 01:08:38.237367 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User: tempest-VolumesNegativeTest-1555677896-project Oct 30 01:08:38.237367 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Domain-Id: default Oct 30 01:08:38.237367 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Domain-Name: Default Oct 30 01:08:38.237367 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Id: 25c6ea71c72747c59d16087ace588111 Oct 30 01:08:38.237367 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Name: tempest-VolumesNegativeTest-1555677896-project Oct 30 01:08:38.237367 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: method: Oct 30 01:08:38.237367 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: exception: Volume tempest-invalid-1906016073 could not be found. Oct 30 01:08:38.237367 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: action_args: {'id': 'tempest-invalid-1906016073'} Oct 30 01:08:38.237367 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 01:08:38.238396 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:38.238396 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 30 01:08:38.238396 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 30 01:08:38.238396 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 30 01:08:38.238396 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 30 01:08:38.238396 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 30 01:08:38.238396 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 30 01:08:38.238396 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 30 01:08:38.238396 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 30 01:08:38.238396 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 30 01:08:38.238396 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 30 01:08:38.238396 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 30 01:08:38.238396 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 30 01:08:38.238396 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 30 01:08:38.238396 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 30 01:08:38.238396 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume tempest-invalid-1906016073 could not be found. (HTTP 404) (Request-ID: req-91ec7ab9-7638-4516-8fb2-cfe51ec0fad8) Oct 30 01:08:38.238396 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:08:38.238396 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:08:38.238396 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:08:38.240005 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:38.240005 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Oct 30 01:08:38.240005 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Oct 30 01:08:38.240005 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 30 01:08:38.240005 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 30 01:08:38.240005 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 30 01:08:38.240005 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 30 01:08:38.240005 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 30 01:08:38.240005 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 30 01:08:38.240005 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 30 01:08:38.240005 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 30 01:08:38.240005 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 30 01:08:38.240005 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 30 01:08:38.240005 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 30 01:08:38.240005 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 30 01:08:38.240005 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 30 01:08:38.240005 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 30 01:08:38.240005 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 30 01:08:38.240005 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 30 01:08:38.242929 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 30 01:08:38.242929 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 30 01:08:38.242929 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 30 01:08:38.242929 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 30 01:08:38.242929 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume tempest-invalid-1906016073 could not be found. Oct 30 01:08:38.242929 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:08:38.242929 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:08:38.242929 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:08:38.242929 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:38.242929 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:08:38.242929 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:08:38.242929 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 01:08:38.242929 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 01:08:38.242929 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 30 01:08:38.242929 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 30 01:08:38.242929 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 01:08:38.242929 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 01:08:38.242929 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Oct 30 01:08:38.242929 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 30 01:08:38.242929 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume tempest-invalid-1906016073 could not be found. Oct 30 01:08:38.244510 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:08:38.244510 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:08:38.244510 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:08:38.244510 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:38.244510 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:08:38.244510 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:08:38.244510 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 01:08:38.244510 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 01:08:38.244510 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume tempest-invalid-1906016073 could not be found. Oct 30 01:08:38.244510 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:08:38.244510 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-51fa2491-40d4-49b9-91dd-019075010000 tempest-VolumesNegativeTest-1555677896 tempest-VolumesNegativeTest-1555677896-project] Returning 404 to user: Volume tempest-invalid-1906016073 could not be found. {{(pid=118961) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 01:08:38.244510 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-51fa2491-40d4-49b9-91dd-019075010000 tempest-VolumesNegativeTest-1555677896 tempest-VolumesNegativeTest-1555677896-project] 10.222.0.16 "DELETE /compute/v2.1/os-volumes/tempest-invalid-1906016073" status: 404 len: 99 microversion: 2.1 time: 0.042646 Oct 30 01:08:38.244510 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 647/1270] 10.222.0.16 () {58 vars in 1218 bytes} [Sun Oct 30 01:08:38 2022] DELETE /compute/v2.1/os-volumes/tempest-invalid-1906016073 => generated 99 bytes in 44 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Oct 30 01:08:38.248494 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-52410597-7faa-4992-94c7-fb8304a50f9c tempest-VolumesNegativeTest-1555677896 tempest-VolumesNegativeTest-1555677896-project] 10.222.0.16 "DELETE /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.004073 Oct 30 01:08:38.248972 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 624/1271] 10.222.0.16 () {58 vars in 1140 bytes} [Sun Oct 30 01:08:38 2022] DELETE /compute/v2.1/os-volumes/ => generated 112 bytes in 7 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Oct 30 01:08:38.258429 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-230d60a0-1306-4e99-9291-ed298a12014b tempest-VolumesNegativeTest-1555677896 tempest-VolumesNegativeTest-1555677896-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.004846 Oct 30 01:08:38.259109 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 648/1272] 10.222.0.16 () {58 vars in 1137 bytes} [Sun Oct 30 01:08:38 2022] GET /compute/v2.1/os-volumes/ => generated 112 bytes in 6 msecs (HTTP/1.1 404) 5 headers in 228 bytes (2 switches on core 0) Oct 30 01:08:38.268242 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-eb3b78e5-e794-4d10-8ff2-d7aa5705b177 tempest-VolumesNegativeTest-1555677896 tempest-VolumesNegativeTest-1555677896-project] Calling method '.version_select at 0x7f0eb13973a0>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:08:38.303619 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.wsgi [None req-eb3b78e5-e794-4d10-8ff2-d7aa5705b177 tempest-VolumesNegativeTest-1555677896 tempest-VolumesNegativeTest-1555677896-project] HTTP exception thrown: Volume e8266162-2e34-421a-9a6d-22d2c4dd4ffa could not be found. Oct 30 01:08:38.305621 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-eb3b78e5-e794-4d10-8ff2-d7aa5705b177 tempest-VolumesNegativeTest-1555677896 tempest-VolumesNegativeTest-1555677896-project] Request method failure captured: Oct 30 01:08:38.305621 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: request: DELETE /compute/v2.1/os-volumes/e8266162-2e34-421a-9a6d-22d2c4dd4ffa HTTP/1.1 Oct 30 01:08:38.305621 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept: application/json Oct 30 01:08:38.305621 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept-Encoding: identity Oct 30 01:08:38.305621 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Connection: close Oct 30 01:08:38.305621 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Length: 0 Oct 30 01:08:38.305621 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Type: application/json Oct 30 01:08:38.305621 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Host: 10.222.0.16 Oct 30 01:08:38.305621 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Openstack-System-Scope: None Oct 30 01:08:38.305621 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: User-Agent: python-urllib3/1.26.12 Oct 30 01:08:38.305621 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Auth-Token: gAAAAABjXc6VrQlE6SHfwoCxrZZqW7NGWSuH_3D_l7GnPJbcH3QXg7CchMG0txgs0Z2hhFPPMw5BhOiE2zzvqaev72qVL5vpi9jQYpYsPSu7ADgn8uTYJHpbz9RJJA-ozO7MnszbCv5ba9nxzoD-BGIwEbqVPQqZWCjwAF8YAhZTWQgF4kZP9Rc Oct 30 01:08:38.305621 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Id: None Oct 30 01:08:38.305621 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Name: None Oct 30 01:08:38.305621 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Identity-Status: Confirmed Oct 30 01:08:38.305621 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Is-Admin-Project: True Oct 30 01:08:38.305621 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Id: default Oct 30 01:08:38.305621 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Name: Default Oct 30 01:08:38.305621 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Id: 4cb13eb3ad574ce9a9150be4773c00fa Oct 30 01:08:38.305621 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Name: tempest-VolumesNegativeTest-1555677896 Oct 30 01:08:38.305621 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Role: reader,member Oct 30 01:08:38.305621 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Roles: reader,member Oct 30 01:08:38.307260 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_4cb13eb3ad574ce9a9150be4773c00fa"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/4cb13eb3ad574ce9a9150be4773c00fa"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/4cb13eb3ad574ce9a9150be4773c00fa"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/4cb13eb3ad574ce9a9150be4773c00fa"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 01:08:38.307260 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant: tempest-VolumesNegativeTest-1555677896 Oct 30 01:08:38.307260 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Id: 4cb13eb3ad574ce9a9150be4773c00fa Oct 30 01:08:38.307260 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Name: tempest-VolumesNegativeTest-1555677896 Oct 30 01:08:38.307260 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User: tempest-VolumesNegativeTest-1555677896-project Oct 30 01:08:38.307260 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Id: default Oct 30 01:08:38.307260 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Name: Default Oct 30 01:08:38.307260 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Id: 25c6ea71c72747c59d16087ace588111 Oct 30 01:08:38.307260 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Name: tempest-VolumesNegativeTest-1555677896-project Oct 30 01:08:38.307260 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: method: Oct 30 01:08:38.307260 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: exception: Volume e8266162-2e34-421a-9a6d-22d2c4dd4ffa could not be found. Oct 30 01:08:38.307260 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: action_args: {'id': 'e8266162-2e34-421a-9a6d-22d2c4dd4ffa'} Oct 30 01:08:38.307260 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 01:08:38.308349 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:38.308349 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 30 01:08:38.308349 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 30 01:08:38.308349 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 30 01:08:38.308349 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 30 01:08:38.308349 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 30 01:08:38.308349 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 30 01:08:38.308349 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 30 01:08:38.308349 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 30 01:08:38.308349 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 30 01:08:38.308349 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 30 01:08:38.308349 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 30 01:08:38.308349 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 30 01:08:38.308349 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 30 01:08:38.308349 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 30 01:08:38.308349 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume e8266162-2e34-421a-9a6d-22d2c4dd4ffa could not be found. (HTTP 404) (Request-ID: req-22cb53a6-9140-49d7-b311-f4cacd384ca9) Oct 30 01:08:38.308349 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:08:38.308349 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:08:38.308349 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:08:38.309906 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:38.309906 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Oct 30 01:08:38.309906 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Oct 30 01:08:38.309906 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 30 01:08:38.309906 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 30 01:08:38.309906 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 30 01:08:38.309906 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 30 01:08:38.309906 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 30 01:08:38.309906 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 30 01:08:38.309906 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 30 01:08:38.309906 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 30 01:08:38.309906 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 30 01:08:38.309906 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 30 01:08:38.309906 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 30 01:08:38.309906 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 30 01:08:38.309906 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 30 01:08:38.309906 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 30 01:08:38.309906 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 30 01:08:38.309906 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 30 01:08:38.311480 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 30 01:08:38.311480 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 30 01:08:38.311480 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 30 01:08:38.311480 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 30 01:08:38.311480 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume e8266162-2e34-421a-9a6d-22d2c4dd4ffa could not be found. Oct 30 01:08:38.311480 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:08:38.311480 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:08:38.311480 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:08:38.311480 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:38.311480 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:08:38.311480 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:08:38.311480 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 01:08:38.311480 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 01:08:38.311480 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 30 01:08:38.311480 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 30 01:08:38.311480 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 01:08:38.311480 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 01:08:38.311480 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Oct 30 01:08:38.311480 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 30 01:08:38.313062 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume e8266162-2e34-421a-9a6d-22d2c4dd4ffa could not be found. Oct 30 01:08:38.313062 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:08:38.313062 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:08:38.313062 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:08:38.313062 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:38.313062 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:08:38.313062 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:08:38.313062 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 01:08:38.313062 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 01:08:38.313062 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume e8266162-2e34-421a-9a6d-22d2c4dd4ffa could not be found. Oct 30 01:08:38.313062 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:08:38.313062 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-eb3b78e5-e794-4d10-8ff2-d7aa5705b177 tempest-VolumesNegativeTest-1555677896 tempest-VolumesNegativeTest-1555677896-project] Returning 404 to user: Volume e8266162-2e34-421a-9a6d-22d2c4dd4ffa could not be found. {{(pid=118962) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 01:08:38.313062 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-eb3b78e5-e794-4d10-8ff2-d7aa5705b177 tempest-VolumesNegativeTest-1555677896 tempest-VolumesNegativeTest-1555677896-project] 10.222.0.16 "DELETE /compute/v2.1/os-volumes/e8266162-2e34-421a-9a6d-22d2c4dd4ffa" status: 404 len: 109 microversion: 2.1 time: 0.042998 Oct 30 01:08:38.313062 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 625/1273] 10.222.0.16 () {58 vars in 1248 bytes} [Sun Oct 30 01:08:38 2022] DELETE /compute/v2.1/os-volumes/e8266162-2e34-421a-9a6d-22d2c4dd4ffa => generated 109 bytes in 48 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 30 01:08:38.319371 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-e56e8cf0-e619-4ef3-bcf9-4cae7a84bc01 tempest-VolumesNegativeTest-1555677896 tempest-VolumesNegativeTest-1555677896-project] Calling method '.version_select at 0x7f0eabaf7c10>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:08:38.348345 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.wsgi [None req-e56e8cf0-e619-4ef3-bcf9-4cae7a84bc01 tempest-VolumesNegativeTest-1555677896 tempest-VolumesNegativeTest-1555677896-project] HTTP exception thrown: Volume 12b1d0ca-da71-4726-968f-e849a0722dc6 could not be found. Oct 30 01:08:38.350311 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-e56e8cf0-e619-4ef3-bcf9-4cae7a84bc01 tempest-VolumesNegativeTest-1555677896 tempest-VolumesNegativeTest-1555677896-project] Request method failure captured: Oct 30 01:08:38.350311 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: request: GET /compute/v2.1/os-volumes/12b1d0ca-da71-4726-968f-e849a0722dc6 HTTP/1.1 Oct 30 01:08:38.350311 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Accept: application/json Oct 30 01:08:38.350311 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Accept-Encoding: identity Oct 30 01:08:38.350311 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Connection: close Oct 30 01:08:38.350311 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Content-Length: 0 Oct 30 01:08:38.350311 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Content-Type: application/json Oct 30 01:08:38.350311 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Host: 10.222.0.16 Oct 30 01:08:38.350311 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Openstack-System-Scope: None Oct 30 01:08:38.350311 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: User-Agent: python-urllib3/1.26.12 Oct 30 01:08:38.350311 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Auth-Token: gAAAAABjXc6VrQlE6SHfwoCxrZZqW7NGWSuH_3D_l7GnPJbcH3QXg7CchMG0txgs0Z2hhFPPMw5BhOiE2zzvqaev72qVL5vpi9jQYpYsPSu7ADgn8uTYJHpbz9RJJA-ozO7MnszbCv5ba9nxzoD-BGIwEbqVPQqZWCjwAF8YAhZTWQgF4kZP9Rc Oct 30 01:08:38.350311 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Domain-Id: None Oct 30 01:08:38.350311 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Domain-Name: None Oct 30 01:08:38.350311 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Identity-Status: Confirmed Oct 30 01:08:38.350311 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Is-Admin-Project: True Oct 30 01:08:38.350311 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Domain-Id: default Oct 30 01:08:38.350311 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Domain-Name: Default Oct 30 01:08:38.350311 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Id: 4cb13eb3ad574ce9a9150be4773c00fa Oct 30 01:08:38.350311 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Name: tempest-VolumesNegativeTest-1555677896 Oct 30 01:08:38.350311 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Role: reader,member Oct 30 01:08:38.350311 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Roles: reader,member Oct 30 01:08:38.351875 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_4cb13eb3ad574ce9a9150be4773c00fa"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/4cb13eb3ad574ce9a9150be4773c00fa"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/4cb13eb3ad574ce9a9150be4773c00fa"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/4cb13eb3ad574ce9a9150be4773c00fa"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 01:08:38.351875 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant: tempest-VolumesNegativeTest-1555677896 Oct 30 01:08:38.351875 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant-Id: 4cb13eb3ad574ce9a9150be4773c00fa Oct 30 01:08:38.351875 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant-Name: tempest-VolumesNegativeTest-1555677896 Oct 30 01:08:38.351875 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User: tempest-VolumesNegativeTest-1555677896-project Oct 30 01:08:38.351875 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Domain-Id: default Oct 30 01:08:38.351875 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Domain-Name: Default Oct 30 01:08:38.351875 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Id: 25c6ea71c72747c59d16087ace588111 Oct 30 01:08:38.351875 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Name: tempest-VolumesNegativeTest-1555677896-project Oct 30 01:08:38.351875 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: method: Oct 30 01:08:38.351875 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: exception: Volume 12b1d0ca-da71-4726-968f-e849a0722dc6 could not be found. Oct 30 01:08:38.351875 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: action_args: {'id': '12b1d0ca-da71-4726-968f-e849a0722dc6'} Oct 30 01:08:38.351875 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 01:08:38.352742 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:38.352742 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 30 01:08:38.352742 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 30 01:08:38.352742 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 30 01:08:38.352742 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 30 01:08:38.352742 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 30 01:08:38.352742 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 30 01:08:38.352742 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 30 01:08:38.352742 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 30 01:08:38.352742 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 30 01:08:38.352742 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 30 01:08:38.352742 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 30 01:08:38.352742 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 30 01:08:38.352742 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 30 01:08:38.352742 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 30 01:08:38.352742 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 12b1d0ca-da71-4726-968f-e849a0722dc6 could not be found. (HTTP 404) (Request-ID: req-a19512e7-44c9-4b0c-afcf-2b7c9281afd9) Oct 30 01:08:38.352742 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:08:38.352742 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:08:38.352742 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:08:38.353977 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:38.353977 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Oct 30 01:08:38.353977 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Oct 30 01:08:38.353977 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 30 01:08:38.353977 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 30 01:08:38.353977 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 30 01:08:38.353977 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 30 01:08:38.353977 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 30 01:08:38.353977 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 30 01:08:38.353977 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 30 01:08:38.353977 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 30 01:08:38.353977 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 30 01:08:38.353977 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 30 01:08:38.353977 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 30 01:08:38.353977 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 30 01:08:38.353977 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 30 01:08:38.353977 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 30 01:08:38.353977 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 30 01:08:38.353977 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 30 01:08:38.355238 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 30 01:08:38.355238 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 30 01:08:38.355238 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 30 01:08:38.355238 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 30 01:08:38.355238 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 12b1d0ca-da71-4726-968f-e849a0722dc6 could not be found. Oct 30 01:08:38.355238 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:08:38.355238 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:08:38.355238 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:08:38.355238 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:38.355238 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:08:38.355238 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:08:38.355238 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 01:08:38.355238 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 01:08:38.355238 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 30 01:08:38.355238 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 30 01:08:38.355238 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 01:08:38.355238 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 01:08:38.355238 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Oct 30 01:08:38.355238 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 30 01:08:38.356521 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 12b1d0ca-da71-4726-968f-e849a0722dc6 could not be found. Oct 30 01:08:38.356521 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:08:38.356521 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:08:38.356521 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:08:38.356521 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:38.356521 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:08:38.356521 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:08:38.356521 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 01:08:38.356521 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 01:08:38.356521 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 12b1d0ca-da71-4726-968f-e849a0722dc6 could not be found. Oct 30 01:08:38.356521 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:08:38.356521 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-e56e8cf0-e619-4ef3-bcf9-4cae7a84bc01 tempest-VolumesNegativeTest-1555677896 tempest-VolumesNegativeTest-1555677896-project] Returning 404 to user: Volume 12b1d0ca-da71-4726-968f-e849a0722dc6 could not be found. {{(pid=118961) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 01:08:38.356521 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-e56e8cf0-e619-4ef3-bcf9-4cae7a84bc01 tempest-VolumesNegativeTest-1555677896 tempest-VolumesNegativeTest-1555677896-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/12b1d0ca-da71-4726-968f-e849a0722dc6" status: 404 len: 109 microversion: 2.1 time: 0.036339 Oct 30 01:08:38.356521 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 649/1274] 10.222.0.16 () {58 vars in 1245 bytes} [Sun Oct 30 01:08:38 2022] GET /compute/v2.1/os-volumes/12b1d0ca-da71-4726-968f-e849a0722dc6 => generated 109 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 30 01:08:38.684756 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-bc6e01dc-0dee-41ce-98b4-1055a5b0fbc1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:08:38.685374 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-bc6e01dc-0dee-41ce-98b4-1055a5b0fbc1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:08:38.692134 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bc6e01dc-0dee-41ce-98b4-1055a5b0fbc1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:38.692536 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bc6e01dc-0dee-41ce-98b4-1055a5b0fbc1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:38.692959 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bc6e01dc-0dee-41ce-98b4-1055a5b0fbc1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:38.693640 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bc6e01dc-0dee-41ce-98b4-1055a5b0fbc1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:38.694084 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bc6e01dc-0dee-41ce-98b4-1055a5b0fbc1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:38.694512 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bc6e01dc-0dee-41ce-98b4-1055a5b0fbc1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:39.079040 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-bc6e01dc-0dee-41ce-98b4-1055a5b0fbc1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:08:39.087856 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-bc6e01dc-0dee-41ce-98b4-1055a5b0fbc1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:08:39.089377 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-bc6e01dc-0dee-41ce-98b4-1055a5b0fbc1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:08:39.090633 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-bc6e01dc-0dee-41ce-98b4-1055a5b0fbc1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1723 microversion: 2.20 time: 0.409860 Oct 30 01:08:39.091267 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 626/1275] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:08:38 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1723 bytes in 411 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:08:39.852356 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [req-beba45de-8050-4f95-adf7-f2e8b84669dc req-69c9e5a9-8513-428e-9c53-480ecf3986bd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f4033e06-e2d3-4abb-ba25-eddfaf3e7e20", "name": "network-vif-deleted", "tag": "e5d64f79-b32b-48ac-8231-fbd53262a835"}]} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:08:39.872580 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-beba45de-8050-4f95-adf7-f2e8b84669dc req-69c9e5a9-8513-428e-9c53-480ecf3986bd service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:39.873358 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-beba45de-8050-4f95-adf7-f2e8b84669dc req-69c9e5a9-8513-428e-9c53-480ecf3986bd service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:39.874156 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-beba45de-8050-4f95-adf7-f2e8b84669dc req-69c9e5a9-8513-428e-9c53-480ecf3986bd service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:39.888701 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.compute.server_external_events [req-beba45de-8050-4f95-adf7-f2e8b84669dc req-69c9e5a9-8513-428e-9c53-480ecf3986bd service nova] Creating event network-vif-deleted:e5d64f79-b32b-48ac-8231-fbd53262a835 for instance f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 on hv-cinder-maste Oct 30 01:08:39.898446 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [req-beba45de-8050-4f95-adf7-f2e8b84669dc req-69c9e5a9-8513-428e-9c53-480ecf3986bd service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.051260 Oct 30 01:08:39.899686 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 650/1276] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:08:39 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 30 01:08:40.115581 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-2d526126-6690-4fc8-bc3e-9ead8207e2e1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:08:40.116253 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-2d526126-6690-4fc8-bc3e-9ead8207e2e1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:08:40.123914 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2d526126-6690-4fc8-bc3e-9ead8207e2e1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:40.124688 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2d526126-6690-4fc8-bc3e-9ead8207e2e1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:40.125327 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2d526126-6690-4fc8-bc3e-9ead8207e2e1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:40.126053 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2d526126-6690-4fc8-bc3e-9ead8207e2e1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:40.126502 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2d526126-6690-4fc8-bc3e-9ead8207e2e1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:40.126924 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2d526126-6690-4fc8-bc3e-9ead8207e2e1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:40.162834 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2d526126-6690-4fc8-bc3e-9ead8207e2e1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:40.163663 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2d526126-6690-4fc8-bc3e-9ead8207e2e1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:40.164281 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2d526126-6690-4fc8-bc3e-9ead8207e2e1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:40.165028 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.objects.instance [None req-2d526126-6690-4fc8-bc3e-9ead8207e2e1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lazy-loading 'fault' on Instance uuid f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 {{(pid=118962) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 30 01:08:40.214831 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-2d526126-6690-4fc8-bc3e-9ead8207e2e1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:08:40.224843 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-2d526126-6690-4fc8-bc3e-9ead8207e2e1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:08:40.226857 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-2d526126-6690-4fc8-bc3e-9ead8207e2e1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '15e603f9898a430d8a53f21818e97722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3bfc7112bb04760aa4e9adc50b12361', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:08:40.227671 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-2d526126-6690-4fc8-bc3e-9ead8207e2e1 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 200 len: 1371 microversion: 2.20 time: 0.116018 Oct 30 01:08:40.228566 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 627/1277] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:08:40 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 1371 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 30 01:08:41.261432 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-3450cb3f-a470-4b90-93ce-95a45ba28318 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:08:41.262153 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-3450cb3f-a470-4b90-93ce-95a45ba28318 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:08:41.270007 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_api:SELECT=42,nova_api:UPDATE=2 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:08:41.278644 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3450cb3f-a470-4b90-93ce-95a45ba28318 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:41.279183 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3450cb3f-a470-4b90-93ce-95a45ba28318 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:41.279619 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3450cb3f-a470-4b90-93ce-95a45ba28318 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:41.280362 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3450cb3f-a470-4b90-93ce-95a45ba28318 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:41.280816 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3450cb3f-a470-4b90-93ce-95a45ba28318 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:41.281232 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3450cb3f-a470-4b90-93ce-95a45ba28318 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:41.303038 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.wsgi [None req-3450cb3f-a470-4b90-93ce-95a45ba28318 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] HTTP exception thrown: Instance f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 could not be found. Oct 30 01:08:41.306432 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-3450cb3f-a470-4b90-93ce-95a45ba28318 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Request method failure captured: Oct 30 01:08:41.306432 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: request: GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 HTTP/1.1 Oct 30 01:08:41.306432 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Accept: application/json Oct 30 01:08:41.306432 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Accept-Encoding: identity Oct 30 01:08:41.306432 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Connection: close Oct 30 01:08:41.306432 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Content-Length: 0 Oct 30 01:08:41.306432 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Content-Type: application/json Oct 30 01:08:41.306432 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Host: 10.222.0.16 Oct 30 01:08:41.306432 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Openstack-System-Scope: None Oct 30 01:08:41.306432 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: User-Agent: python-urllib3/1.26.12 Oct 30 01:08:41.306432 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Auth-Token: gAAAAABjXcugVWKyquPgiWxxmLIR7tjn5wgo5PMehRN68UdChNzQYP0dt_zzBfBJguIoFG_dA8xfdVXJfHfs12PVvciy_RYdtFnj0iZgJFLLfVRiSCDdkIwZN-XNhM2aC5XCKhmUA8zgKICHzu0rW4fXBpKKyrnf0oul0CZJ7cPc4A8OWNhd6Jg Oct 30 01:08:41.306432 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Domain-Id: None Oct 30 01:08:41.306432 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Domain-Name: None Oct 30 01:08:41.306432 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Identity-Status: Confirmed Oct 30 01:08:41.306432 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Is-Admin-Project: True Oct 30 01:08:41.306432 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Openstack-Nova-Api-Version: 2.20 Oct 30 01:08:41.306432 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Domain-Id: default Oct 30 01:08:41.306432 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Domain-Name: Default Oct 30 01:08:41.306432 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Id: a3bfc7112bb04760aa4e9adc50b12361 Oct 30 01:08:41.306432 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-40368566 Oct 30 01:08:41.306432 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Role: reader,member Oct 30 01:08:41.306432 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Roles: reader,member Oct 30 01:08:41.308344 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_a3bfc7112bb04760aa4e9adc50b12361"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a3bfc7112bb04760aa4e9adc50b12361"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/a3bfc7112bb04760aa4e9adc50b12361"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a3bfc7112bb04760aa4e9adc50b12361"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 01:08:41.308344 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant: tempest-AttachVolumeShelveTestJSON-40368566 Oct 30 01:08:41.308344 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant-Id: a3bfc7112bb04760aa4e9adc50b12361 Oct 30 01:08:41.308344 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-40368566 Oct 30 01:08:41.308344 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User: tempest-AttachVolumeShelveTestJSON-40368566-project Oct 30 01:08:41.308344 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Domain-Id: default Oct 30 01:08:41.308344 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Domain-Name: Default Oct 30 01:08:41.308344 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Id: 15e603f9898a430d8a53f21818e97722 Oct 30 01:08:41.308344 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Name: tempest-AttachVolumeShelveTestJSON-40368566-project Oct 30 01:08:41.308344 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: method: > Oct 30 01:08:41.308344 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: exception: Instance f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 could not be found. Oct 30 01:08:41.308344 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: action_args: {'id': 'f4033e06-e2d3-4abb-ba25-eddfaf3e7e20'} Oct 30 01:08:41.309940 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 01:08:41.309940 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:41.309940 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 30 01:08:41.309940 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 30 01:08:41.309940 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 30 01:08:41.309940 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 30 01:08:41.309940 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 30 01:08:41.309940 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 30 01:08:41.309940 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 30 01:08:41.309940 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 30 01:08:41.309940 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 could not be found. Oct 30 01:08:41.309940 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:08:41.309940 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:08:41.309940 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:08:41.309940 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:41.309940 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:08:41.309940 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:08:41.309940 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 01:08:41.309940 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 01:08:41.309940 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 01:08:41.311440 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 01:08:41.311440 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 30 01:08:41.311440 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 30 01:08:41.311440 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 30 01:08:41.311440 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 30 01:08:41.311440 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 30 01:08:41.311440 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 30 01:08:41.311440 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 could not be found. Oct 30 01:08:41.311440 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:08:41.311440 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:08:41.311440 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:08:41.311440 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:41.311440 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:08:41.311440 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:08:41.311440 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 01:08:41.311440 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 01:08:41.311440 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 could not be found. Oct 30 01:08:41.311440 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:08:41.314029 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-3450cb3f-a470-4b90-93ce-95a45ba28318 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Returning 404 to user: Instance f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 could not be found. {{(pid=118961) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 01:08:41.314029 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-3450cb3f-a470-4b90-93ce-95a45ba28318 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 404 len: 111 microversion: 2.20 time: 0.058543 Oct 30 01:08:41.314029 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 651/1278] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:08:41 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 111 bytes in 60 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Oct 30 01:08:41.317589 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-5d7908c9-b5c2-4ca1-bbc0-db5368201e79 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '.version_select at 0x7f0eb09d1b80>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:08:41.340755 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-5d7908c9-b5c2-4ca1-bbc0-db5368201e79 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-872797247" status: 204 len: 0 microversion: 2.20 time: 0.026929 Oct 30 01:08:41.345272 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 628/1279] 10.222.0.16 () {60 vars in 1259 bytes} [Sun Oct 30 01:08:41 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-872797247 => generated 0 bytes in 29 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 30 01:08:43.337232 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-d6238d29-45f1-45fa-ba83-b7668e2203ae tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:08:43.337735 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-d6238d29-45f1-45fa-ba83-b7668e2203ae tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:08:43.344786 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d6238d29-45f1-45fa-ba83-b7668e2203ae tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:43.345656 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d6238d29-45f1-45fa-ba83-b7668e2203ae tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:43.346276 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d6238d29-45f1-45fa-ba83-b7668e2203ae tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:43.347153 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d6238d29-45f1-45fa-ba83-b7668e2203ae tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:43.347846 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d6238d29-45f1-45fa-ba83-b7668e2203ae tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:43.348418 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d6238d29-45f1-45fa-ba83-b7668e2203ae tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:43.363448 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.wsgi [None req-d6238d29-45f1-45fa-ba83-b7668e2203ae tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] HTTP exception thrown: Instance f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 could not be found. Oct 30 01:08:43.373838 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-d6238d29-45f1-45fa-ba83-b7668e2203ae tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Request method failure captured: Oct 30 01:08:43.373838 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: request: DELETE /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 HTTP/1.1 Oct 30 01:08:43.373838 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Accept: application/json Oct 30 01:08:43.373838 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Accept-Encoding: identity Oct 30 01:08:43.373838 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Connection: close Oct 30 01:08:43.373838 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Content-Length: 0 Oct 30 01:08:43.373838 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Content-Type: application/json Oct 30 01:08:43.373838 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Host: 10.222.0.16 Oct 30 01:08:43.373838 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Openstack-System-Scope: None Oct 30 01:08:43.373838 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: User-Agent: python-urllib3/1.26.12 Oct 30 01:08:43.373838 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Auth-Token: gAAAAABjXcugVWKyquPgiWxxmLIR7tjn5wgo5PMehRN68UdChNzQYP0dt_zzBfBJguIoFG_dA8xfdVXJfHfs12PVvciy_RYdtFnj0iZgJFLLfVRiSCDdkIwZN-XNhM2aC5XCKhmUA8zgKICHzu0rW4fXBpKKyrnf0oul0CZJ7cPc4A8OWNhd6Jg Oct 30 01:08:43.373838 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Domain-Id: None Oct 30 01:08:43.373838 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Domain-Name: None Oct 30 01:08:43.373838 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Identity-Status: Confirmed Oct 30 01:08:43.373838 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Is-Admin-Project: True Oct 30 01:08:43.373838 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Openstack-Nova-Api-Version: 2.20 Oct 30 01:08:43.373838 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Domain-Id: default Oct 30 01:08:43.373838 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Domain-Name: Default Oct 30 01:08:43.373838 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Id: a3bfc7112bb04760aa4e9adc50b12361 Oct 30 01:08:43.373838 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-40368566 Oct 30 01:08:43.373838 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Role: reader,member Oct 30 01:08:43.373838 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Roles: reader,member Oct 30 01:08:43.375528 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_a3bfc7112bb04760aa4e9adc50b12361"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a3bfc7112bb04760aa4e9adc50b12361"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/a3bfc7112bb04760aa4e9adc50b12361"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a3bfc7112bb04760aa4e9adc50b12361"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 01:08:43.375528 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant: tempest-AttachVolumeShelveTestJSON-40368566 Oct 30 01:08:43.375528 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant-Id: a3bfc7112bb04760aa4e9adc50b12361 Oct 30 01:08:43.375528 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-40368566 Oct 30 01:08:43.375528 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User: tempest-AttachVolumeShelveTestJSON-40368566-project Oct 30 01:08:43.375528 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Domain-Id: default Oct 30 01:08:43.375528 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Domain-Name: Default Oct 30 01:08:43.375528 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Id: 15e603f9898a430d8a53f21818e97722 Oct 30 01:08:43.375528 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Name: tempest-AttachVolumeShelveTestJSON-40368566-project Oct 30 01:08:43.375528 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: method: > Oct 30 01:08:43.375528 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: exception: Instance f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 could not be found. Oct 30 01:08:43.375528 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: action_args: {'id': 'f4033e06-e2d3-4abb-ba25-eddfaf3e7e20'} Oct 30 01:08:43.376756 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 01:08:43.376756 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:43.376756 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 30 01:08:43.376756 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 30 01:08:43.376756 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 30 01:08:43.376756 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 30 01:08:43.376756 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 30 01:08:43.376756 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 30 01:08:43.376756 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 30 01:08:43.376756 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 30 01:08:43.376756 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 could not be found. Oct 30 01:08:43.376756 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:08:43.376756 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:08:43.376756 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:08:43.376756 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:43.376756 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:08:43.376756 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:08:43.376756 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 01:08:43.376756 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 01:08:43.376756 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 01:08:43.378334 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 01:08:43.378334 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 30 01:08:43.378334 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 30 01:08:43.378334 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 30 01:08:43.378334 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 30 01:08:43.378334 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 30 01:08:43.378334 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 30 01:08:43.378334 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 30 01:08:43.378334 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 30 01:08:43.378334 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 could not be found. Oct 30 01:08:43.378334 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:08:43.378334 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:08:43.378334 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:08:43.378334 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:43.378334 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:08:43.378334 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:08:43.378334 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 01:08:43.378334 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 01:08:43.378334 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 could not be found. Oct 30 01:08:43.378334 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:08:43.380812 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-d6238d29-45f1-45fa-ba83-b7668e2203ae tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Returning 404 to user: Instance f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 could not be found. {{(pid=118961) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 01:08:43.380812 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-d6238d29-45f1-45fa-ba83-b7668e2203ae tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "DELETE /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 404 len: 111 microversion: 2.20 time: 0.033760 Oct 30 01:08:43.380812 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 652/1280] 10.222.0.16 () {60 vars in 1280 bytes} [Sun Oct 30 01:08:43 2022] DELETE /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 30 01:08:43.381064 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-6e0f55ec-2feb-4f68-87b4-21b126131c7b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:08:43.381064 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-6e0f55ec-2feb-4f68-87b4-21b126131c7b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: f4033e06-e2d3-4abb-ba25-eddfaf3e7e20] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:08:43.384384 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6e0f55ec-2feb-4f68-87b4-21b126131c7b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:43.384809 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6e0f55ec-2feb-4f68-87b4-21b126131c7b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:43.385241 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6e0f55ec-2feb-4f68-87b4-21b126131c7b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:43.385923 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6e0f55ec-2feb-4f68-87b4-21b126131c7b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:43.386364 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6e0f55ec-2feb-4f68-87b4-21b126131c7b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:43.386774 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6e0f55ec-2feb-4f68-87b4-21b126131c7b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:43.400015 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.wsgi [None req-6e0f55ec-2feb-4f68-87b4-21b126131c7b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] HTTP exception thrown: Instance f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 could not be found. Oct 30 01:08:43.401563 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-6e0f55ec-2feb-4f68-87b4-21b126131c7b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Request method failure captured: Oct 30 01:08:43.401563 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: request: GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 HTTP/1.1 Oct 30 01:08:43.401563 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept: application/json Oct 30 01:08:43.401563 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept-Encoding: identity Oct 30 01:08:43.401563 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Connection: close Oct 30 01:08:43.401563 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Length: 0 Oct 30 01:08:43.401563 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Type: application/json Oct 30 01:08:43.401563 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Host: 10.222.0.16 Oct 30 01:08:43.401563 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Openstack-System-Scope: None Oct 30 01:08:43.401563 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: User-Agent: python-urllib3/1.26.12 Oct 30 01:08:43.401563 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Auth-Token: gAAAAABjXcugVWKyquPgiWxxmLIR7tjn5wgo5PMehRN68UdChNzQYP0dt_zzBfBJguIoFG_dA8xfdVXJfHfs12PVvciy_RYdtFnj0iZgJFLLfVRiSCDdkIwZN-XNhM2aC5XCKhmUA8zgKICHzu0rW4fXBpKKyrnf0oul0CZJ7cPc4A8OWNhd6Jg Oct 30 01:08:43.401563 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Id: None Oct 30 01:08:43.401563 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Name: None Oct 30 01:08:43.401563 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Identity-Status: Confirmed Oct 30 01:08:43.401563 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Is-Admin-Project: True Oct 30 01:08:43.401563 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Openstack-Nova-Api-Version: 2.20 Oct 30 01:08:43.401563 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Id: default Oct 30 01:08:43.401563 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Name: Default Oct 30 01:08:43.401563 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Id: a3bfc7112bb04760aa4e9adc50b12361 Oct 30 01:08:43.401563 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-40368566 Oct 30 01:08:43.401563 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Role: reader,member Oct 30 01:08:43.401563 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Roles: reader,member Oct 30 01:08:43.403745 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_a3bfc7112bb04760aa4e9adc50b12361"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a3bfc7112bb04760aa4e9adc50b12361"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/a3bfc7112bb04760aa4e9adc50b12361"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a3bfc7112bb04760aa4e9adc50b12361"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 01:08:43.403745 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant: tempest-AttachVolumeShelveTestJSON-40368566 Oct 30 01:08:43.403745 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Id: a3bfc7112bb04760aa4e9adc50b12361 Oct 30 01:08:43.403745 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-40368566 Oct 30 01:08:43.403745 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User: tempest-AttachVolumeShelveTestJSON-40368566-project Oct 30 01:08:43.403745 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Id: default Oct 30 01:08:43.403745 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Name: Default Oct 30 01:08:43.403745 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Id: 15e603f9898a430d8a53f21818e97722 Oct 30 01:08:43.403745 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Name: tempest-AttachVolumeShelveTestJSON-40368566-project Oct 30 01:08:43.403745 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: method: > Oct 30 01:08:43.403745 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: exception: Instance f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 could not be found. Oct 30 01:08:43.403745 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: action_args: {'id': 'f4033e06-e2d3-4abb-ba25-eddfaf3e7e20'} Oct 30 01:08:43.404849 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 01:08:43.404849 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:43.404849 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 30 01:08:43.404849 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 30 01:08:43.404849 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 30 01:08:43.404849 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 30 01:08:43.404849 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 30 01:08:43.404849 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 30 01:08:43.404849 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 30 01:08:43.404849 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 30 01:08:43.404849 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 could not be found. Oct 30 01:08:43.404849 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:08:43.404849 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:08:43.404849 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:08:43.404849 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:43.404849 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:08:43.404849 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:08:43.404849 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 01:08:43.404849 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 01:08:43.404849 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 01:08:43.406343 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 01:08:43.406343 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 30 01:08:43.406343 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 30 01:08:43.406343 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 30 01:08:43.406343 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 30 01:08:43.406343 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 30 01:08:43.406343 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 30 01:08:43.406343 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 could not be found. Oct 30 01:08:43.406343 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:08:43.406343 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:08:43.406343 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:08:43.406343 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:43.406343 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:08:43.406343 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:08:43.406343 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 01:08:43.406343 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 01:08:43.406343 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 could not be found. Oct 30 01:08:43.406343 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:08:43.408011 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-6e0f55ec-2feb-4f68-87b4-21b126131c7b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Returning 404 to user: Instance f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 could not be found. {{(pid=118962) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 01:08:43.408011 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-6e0f55ec-2feb-4f68-87b4-21b126131c7b tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20" status: 404 len: 111 microversion: 2.20 time: 0.031421 Oct 30 01:08:43.408011 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 629/1281] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:08:43 2022] GET /compute/v2.1/servers/f4033e06-e2d3-4abb-ba25-eddfaf3e7e20 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 30 01:08:44.552596 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-d8d78544-fa3f-442d-870b-c4874a20d0f2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:08:44.553102 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-d8d78544-fa3f-442d-870b-c4874a20d0f2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:08:44.560020 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d8d78544-fa3f-442d-870b-c4874a20d0f2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:44.560506 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d8d78544-fa3f-442d-870b-c4874a20d0f2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:44.560940 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d8d78544-fa3f-442d-870b-c4874a20d0f2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:44.561633 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d8d78544-fa3f-442d-870b-c4874a20d0f2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:44.562111 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d8d78544-fa3f-442d-870b-c4874a20d0f2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:44.562519 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d8d78544-fa3f-442d-870b-c4874a20d0f2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:44.576553 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.wsgi [None req-d8d78544-fa3f-442d-870b-c4874a20d0f2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] HTTP exception thrown: Instance 9491a8f4-2b76-4a4e-a6bd-6720114e482f could not be found. Oct 30 01:08:44.578395 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-d8d78544-fa3f-442d-870b-c4874a20d0f2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Request method failure captured: Oct 30 01:08:44.578395 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: request: DELETE /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f HTTP/1.1 Oct 30 01:08:44.578395 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Accept: application/json Oct 30 01:08:44.578395 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Accept-Encoding: identity Oct 30 01:08:44.578395 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Connection: close Oct 30 01:08:44.578395 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Content-Length: 0 Oct 30 01:08:44.578395 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Content-Type: application/json Oct 30 01:08:44.578395 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Host: 10.222.0.16 Oct 30 01:08:44.578395 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Openstack-System-Scope: None Oct 30 01:08:44.578395 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: User-Agent: python-urllib3/1.26.12 Oct 30 01:08:44.578395 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Auth-Token: gAAAAABjXcugVWKyquPgiWxxmLIR7tjn5wgo5PMehRN68UdChNzQYP0dt_zzBfBJguIoFG_dA8xfdVXJfHfs12PVvciy_RYdtFnj0iZgJFLLfVRiSCDdkIwZN-XNhM2aC5XCKhmUA8zgKICHzu0rW4fXBpKKyrnf0oul0CZJ7cPc4A8OWNhd6Jg Oct 30 01:08:44.578395 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Domain-Id: None Oct 30 01:08:44.578395 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Domain-Name: None Oct 30 01:08:44.578395 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Identity-Status: Confirmed Oct 30 01:08:44.578395 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Is-Admin-Project: True Oct 30 01:08:44.578395 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Openstack-Nova-Api-Version: 2.20 Oct 30 01:08:44.578395 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Domain-Id: default Oct 30 01:08:44.578395 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Domain-Name: Default Oct 30 01:08:44.578395 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Id: a3bfc7112bb04760aa4e9adc50b12361 Oct 30 01:08:44.578395 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-40368566 Oct 30 01:08:44.578395 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Role: reader,member Oct 30 01:08:44.578395 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Roles: reader,member Oct 30 01:08:44.580118 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_a3bfc7112bb04760aa4e9adc50b12361"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a3bfc7112bb04760aa4e9adc50b12361"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/a3bfc7112bb04760aa4e9adc50b12361"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a3bfc7112bb04760aa4e9adc50b12361"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 01:08:44.580118 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant: tempest-AttachVolumeShelveTestJSON-40368566 Oct 30 01:08:44.580118 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant-Id: a3bfc7112bb04760aa4e9adc50b12361 Oct 30 01:08:44.580118 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-40368566 Oct 30 01:08:44.580118 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User: tempest-AttachVolumeShelveTestJSON-40368566-project Oct 30 01:08:44.580118 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Domain-Id: default Oct 30 01:08:44.580118 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Domain-Name: Default Oct 30 01:08:44.580118 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Id: 15e603f9898a430d8a53f21818e97722 Oct 30 01:08:44.580118 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Name: tempest-AttachVolumeShelveTestJSON-40368566-project Oct 30 01:08:44.580118 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: method: > Oct 30 01:08:44.580118 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: exception: Instance 9491a8f4-2b76-4a4e-a6bd-6720114e482f could not be found. Oct 30 01:08:44.580118 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: action_args: {'id': '9491a8f4-2b76-4a4e-a6bd-6720114e482f'} Oct 30 01:08:44.581073 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 01:08:44.581073 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:44.581073 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 30 01:08:44.581073 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 30 01:08:44.581073 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 30 01:08:44.581073 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 30 01:08:44.581073 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 30 01:08:44.581073 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 30 01:08:44.581073 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 30 01:08:44.581073 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 30 01:08:44.581073 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9491a8f4-2b76-4a4e-a6bd-6720114e482f could not be found. Oct 30 01:08:44.581073 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:08:44.581073 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:08:44.581073 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:08:44.581073 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:44.581073 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:08:44.581073 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:08:44.581073 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 01:08:44.581073 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 01:08:44.581073 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 01:08:44.582639 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 01:08:44.582639 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 30 01:08:44.582639 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 30 01:08:44.582639 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 30 01:08:44.582639 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 30 01:08:44.582639 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 30 01:08:44.582639 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 30 01:08:44.582639 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 30 01:08:44.582639 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 30 01:08:44.582639 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9491a8f4-2b76-4a4e-a6bd-6720114e482f could not be found. Oct 30 01:08:44.582639 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:08:44.582639 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:08:44.582639 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:08:44.582639 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:44.582639 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:08:44.582639 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:08:44.582639 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 01:08:44.582639 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 01:08:44.582639 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9491a8f4-2b76-4a4e-a6bd-6720114e482f could not be found. Oct 30 01:08:44.582639 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:08:44.584290 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-d8d78544-fa3f-442d-870b-c4874a20d0f2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Returning 404 to user: Instance 9491a8f4-2b76-4a4e-a6bd-6720114e482f could not be found. {{(pid=118961) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 01:08:44.584290 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-d8d78544-fa3f-442d-870b-c4874a20d0f2 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "DELETE /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 404 len: 111 microversion: 2.20 time: 0.033043 Oct 30 01:08:44.586496 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 653/1282] 10.222.0.16 () {60 vars in 1280 bytes} [Sun Oct 30 01:08:44 2022] DELETE /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 30 01:08:44.591827 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-8b381472-ab8b-4fd2-aaef-aeb33fb92885 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:08:44.592428 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-8b381472-ab8b-4fd2-aaef-aeb33fb92885 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] [instance: 9491a8f4-2b76-4a4e-a6bd-6720114e482f] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:08:44.599083 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8b381472-ab8b-4fd2-aaef-aeb33fb92885 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:44.599564 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8b381472-ab8b-4fd2-aaef-aeb33fb92885 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:44.600086 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8b381472-ab8b-4fd2-aaef-aeb33fb92885 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:44.600779 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8b381472-ab8b-4fd2-aaef-aeb33fb92885 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:08:44.601223 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8b381472-ab8b-4fd2-aaef-aeb33fb92885 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:08:44.601638 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8b381472-ab8b-4fd2-aaef-aeb33fb92885 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:08:44.616506 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.wsgi [None req-8b381472-ab8b-4fd2-aaef-aeb33fb92885 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] HTTP exception thrown: Instance 9491a8f4-2b76-4a4e-a6bd-6720114e482f could not be found. Oct 30 01:08:44.618087 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-8b381472-ab8b-4fd2-aaef-aeb33fb92885 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Request method failure captured: Oct 30 01:08:44.618087 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: request: GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f HTTP/1.1 Oct 30 01:08:44.618087 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept: application/json Oct 30 01:08:44.618087 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept-Encoding: identity Oct 30 01:08:44.618087 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Connection: close Oct 30 01:08:44.618087 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Length: 0 Oct 30 01:08:44.618087 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Type: application/json Oct 30 01:08:44.618087 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Host: 10.222.0.16 Oct 30 01:08:44.618087 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Openstack-System-Scope: None Oct 30 01:08:44.618087 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: User-Agent: python-urllib3/1.26.12 Oct 30 01:08:44.618087 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Auth-Token: gAAAAABjXcugVWKyquPgiWxxmLIR7tjn5wgo5PMehRN68UdChNzQYP0dt_zzBfBJguIoFG_dA8xfdVXJfHfs12PVvciy_RYdtFnj0iZgJFLLfVRiSCDdkIwZN-XNhM2aC5XCKhmUA8zgKICHzu0rW4fXBpKKyrnf0oul0CZJ7cPc4A8OWNhd6Jg Oct 30 01:08:44.618087 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Id: None Oct 30 01:08:44.618087 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Name: None Oct 30 01:08:44.618087 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Identity-Status: Confirmed Oct 30 01:08:44.618087 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Is-Admin-Project: True Oct 30 01:08:44.618087 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Openstack-Nova-Api-Version: 2.20 Oct 30 01:08:44.618087 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Id: default Oct 30 01:08:44.618087 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Name: Default Oct 30 01:08:44.618087 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Id: a3bfc7112bb04760aa4e9adc50b12361 Oct 30 01:08:44.618087 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-40368566 Oct 30 01:08:44.618087 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Role: reader,member Oct 30 01:08:44.618087 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Roles: reader,member Oct 30 01:08:44.619640 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_a3bfc7112bb04760aa4e9adc50b12361"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a3bfc7112bb04760aa4e9adc50b12361"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/a3bfc7112bb04760aa4e9adc50b12361"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a3bfc7112bb04760aa4e9adc50b12361"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 01:08:44.619640 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant: tempest-AttachVolumeShelveTestJSON-40368566 Oct 30 01:08:44.619640 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Id: a3bfc7112bb04760aa4e9adc50b12361 Oct 30 01:08:44.619640 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-40368566 Oct 30 01:08:44.619640 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User: tempest-AttachVolumeShelveTestJSON-40368566-project Oct 30 01:08:44.619640 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Id: default Oct 30 01:08:44.619640 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Name: Default Oct 30 01:08:44.619640 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Id: 15e603f9898a430d8a53f21818e97722 Oct 30 01:08:44.619640 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Name: tempest-AttachVolumeShelveTestJSON-40368566-project Oct 30 01:08:44.619640 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: method: > Oct 30 01:08:44.619640 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: exception: Instance 9491a8f4-2b76-4a4e-a6bd-6720114e482f could not be found. Oct 30 01:08:44.619640 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: action_args: {'id': '9491a8f4-2b76-4a4e-a6bd-6720114e482f'} Oct 30 01:08:44.621236 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 01:08:44.621236 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:44.621236 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 30 01:08:44.621236 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 30 01:08:44.621236 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 30 01:08:44.621236 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 30 01:08:44.621236 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 30 01:08:44.621236 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 30 01:08:44.621236 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 30 01:08:44.621236 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 30 01:08:44.621236 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9491a8f4-2b76-4a4e-a6bd-6720114e482f could not be found. Oct 30 01:08:44.621236 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:08:44.621236 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:08:44.621236 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:08:44.621236 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:44.621236 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:08:44.621236 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:08:44.621236 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 01:08:44.621236 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 01:08:44.621236 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 01:08:44.623371 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 01:08:44.623371 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 30 01:08:44.623371 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 30 01:08:44.623371 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 30 01:08:44.623371 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 30 01:08:44.623371 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 30 01:08:44.623371 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 30 01:08:44.623371 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9491a8f4-2b76-4a4e-a6bd-6720114e482f could not be found. Oct 30 01:08:44.623371 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:08:44.623371 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:08:44.623371 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:08:44.623371 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:08:44.623371 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:08:44.623371 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:08:44.623371 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 01:08:44.623371 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 01:08:44.623371 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9491a8f4-2b76-4a4e-a6bd-6720114e482f could not be found. Oct 30 01:08:44.623371 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:08:44.624916 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-8b381472-ab8b-4fd2-aaef-aeb33fb92885 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] Returning 404 to user: Instance 9491a8f4-2b76-4a4e-a6bd-6720114e482f could not be found. {{(pid=118962) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 01:08:44.624916 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-8b381472-ab8b-4fd2-aaef-aeb33fb92885 tempest-AttachVolumeShelveTestJSON-40368566 tempest-AttachVolumeShelveTestJSON-40368566-project] 10.222.0.16 "GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f" status: 404 len: 111 microversion: 2.20 time: 0.031680 Oct 30 01:08:44.624916 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 630/1283] 10.222.0.16 () {60 vars in 1277 bytes} [Sun Oct 30 01:08:44 2022] GET /compute/v2.1/servers/9491a8f4-2b76-4a4e-a6bd-6720114e482f => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 30 01:09:41.902877 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_api:SELECT=5 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:41.903628 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell1:SELECT=69,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:42.072183 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-5d52f19f-e937-4c30-8bc9-bdd922bf06d7 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Action: 'create', calling method: .version_select at 0x7f0eab9a0ee0>, body: {"keypair": {"name": "tempest-TestShelveInstance-1365136734", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHRhmMMmN3NzXhLJRkj833paPmzFv1aCmew1LpJrycFYq2s5ZkkNWZp4dk5kW+RWjPNPolrw9nC5G0eqooBACtKX6BxDXwDdXA86jFiB/FWOfGbVVhRbJ9RLVuuMlFYdrw=="}} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:09:42.088494 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.quota [None req-5d52f19f-e937-4c30-8bc9-bdd922bf06d7 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Getting quotas for project 4f281bf8af484f7a88df7dc8ac06d887. Resources: {'key_pairs'} {{(pid=118961) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 30 01:09:42.092428 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.quota [None req-5d52f19f-e937-4c30-8bc9-bdd922bf06d7 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Getting quotas for user baeaad0029f1455497fc3aea467dfc47 and project 4f281bf8af484f7a88df7dc8ac06d887. Resources: {'key_pairs'} {{(pid=118961) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 30 01:09:42.104448 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR oslo.messaging._drivers.impl_rabbit [None req-5d52f19f-e937-4c30-8bc9-bdd922bf06d7 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [860fac52-48ee-4cd0-84ba-62dff04f4f58] AMQP server on 10.222.0.16:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Oct 30 01:09:43.112560 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO oslo.messaging._drivers.impl_rabbit [None req-5d52f19f-e937-4c30-8bc9-bdd922bf06d7 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [860fac52-48ee-4cd0-84ba-62dff04f4f58] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 34834. Oct 30 01:09:43.129741 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-5d52f19f-e937-4c30-8bc9-bdd922bf06d7 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 397 microversion: 2.1 time: 1.231749 Oct 30 01:09:43.130331 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 654/1284] 10.222.0.16 () {60 vars in 1159 bytes} [Sun Oct 30 01:09:41 2022] POST /compute/v2.1/os-keypairs => generated 397 bytes in 1233 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:10:05.833706 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_api:SELECT=20,nova_api:DELETE=1 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:05.834452 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell1:SELECT=55 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:05.835366 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 30 01:10:05.835544 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Sun Oct 30 01:10:05 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.16) !!! Oct 30 01:10:05.842801 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-b0506c95-084f-4a28-b9c4-9f66ce191082 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestShelveInstance-server-1610007410", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestShelveInstance-1365136734", "security_groups": [{"name": "tempest-secgroup-smoke-327237405"}], "networks": [{"uuid": "c4f8d00c-b75f-4bb8-966c-dae443b9ddea"}], "block_device_mapping_v2": [{"uuid": "51266605-ea88-4db1-b59e-17135a0863c2", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}]}} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:10:05.968177 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.network.neutron [None req-b0506c95-084f-4a28-b9c4-9f66ce191082 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] validate_networks() for [('c4f8d00c-b75f-4bb8-966c-dae443b9ddea', None, None, None, None, None)] {{(pid=118962) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 30 01:10:06.628115 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.virt.hardware [None req-b0506c95-084f-4a28-b9c4-9f66ce191082 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Flavor limits 0:0:0 {{(pid=118962) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 30 01:10:06.628634 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.virt.hardware [None req-b0506c95-084f-4a28-b9c4-9f66ce191082 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Image limits 0:0:0 {{(pid=118962) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 30 01:10:06.629133 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.virt.hardware [None req-b0506c95-084f-4a28-b9c4-9f66ce191082 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Flavor pref 0:0:0 {{(pid=118962) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 30 01:10:06.629540 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.virt.hardware [None req-b0506c95-084f-4a28-b9c4-9f66ce191082 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Image pref 0:0:0 {{(pid=118962) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 30 01:10:06.629945 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.virt.hardware [None req-b0506c95-084f-4a28-b9c4-9f66ce191082 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118962) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 30 01:10:06.639519 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b0506c95-084f-4a28-b9c4-9f66ce191082 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:06.640037 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b0506c95-084f-4a28-b9c4-9f66ce191082 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:06.640471 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b0506c95-084f-4a28-b9c4-9f66ce191082 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:06.641202 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b0506c95-084f-4a28-b9c4-9f66ce191082 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:06.641686 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b0506c95-084f-4a28-b9c4-9f66ce191082 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:06.642138 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b0506c95-084f-4a28-b9c4-9f66ce191082 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:06.659935 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.quota [None req-b0506c95-084f-4a28-b9c4-9f66ce191082 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Getting quotas for project 4f281bf8af484f7a88df7dc8ac06d887. Resources: {'ram', 'instances', 'cores'} {{(pid=118962) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 30 01:10:06.664376 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.quota [None req-b0506c95-084f-4a28-b9c4-9f66ce191082 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Getting quotas for user baeaad0029f1455497fc3aea467dfc47 and project 4f281bf8af484f7a88df7dc8ac06d887. Resources: {'ram', 'instances', 'cores'} {{(pid=118962) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 30 01:10:06.723641 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-b0506c95-084f-4a28-b9c4-9f66ce191082 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Going to run 1 instances... {{(pid=118962) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 30 01:10:06.725797 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-b0506c95-084f-4a28-b9c4-9f66ce191082 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='51266605-ea88-4db1-b59e-17135a0863c2',volume_size=None,volume_type=None)] {{(pid=118962) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 30 01:10:06.867356 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-b0506c95-084f-4a28-b9c4-9f66ce191082 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 1.035765 Oct 30 01:10:06.874157 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 631/1285] 10.222.0.16 () {60 vars in 1147 bytes} [Sun Oct 30 01:10:05 2022] POST /compute/v2.1/servers => generated 405 bytes in 1037 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 30 01:10:06.877254 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Too many heartbeats missed Oct 30 01:10:06.878412 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell0:SELECT=4 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:06.879154 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_api:SELECT=11,nova_api:INSERT=1 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:06.884631 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-07523275-e19b-44a7-b6f4-b359e8fa6169 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:10:06.885243 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-07523275-e19b-44a7-b6f4-b359e8fa6169 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:10:06.944278 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-07523275-e19b-44a7-b6f4-b359e8fa6169 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baeaad0029f1455497fc3aea467dfc47', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f281bf8af484f7a88df7dc8ac06d887', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:10:06.951622 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-07523275-e19b-44a7-b6f4-b359e8fa6169 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5" status: 200 len: 1089 microversion: 2.1 time: 0.076214 Oct 30 01:10:06.952223 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 655/1286] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:10:06 2022] GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5 => generated 1089 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:10:07.976058 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-7870166a-4f6b-4bed-82ea-c9557eab9bb2 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:10:07.976575 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-7870166a-4f6b-4bed-82ea-c9557eab9bb2 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:10:07.983366 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7870166a-4f6b-4bed-82ea-c9557eab9bb2 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:07.983873 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7870166a-4f6b-4bed-82ea-c9557eab9bb2 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:07.984304 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7870166a-4f6b-4bed-82ea-c9557eab9bb2 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:07.984995 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7870166a-4f6b-4bed-82ea-c9557eab9bb2 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:07.985437 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7870166a-4f6b-4bed-82ea-c9557eab9bb2 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:07.985852 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7870166a-4f6b-4bed-82ea-c9557eab9bb2 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:08.052599 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-7870166a-4f6b-4bed-82ea-c9557eab9bb2 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baeaad0029f1455497fc3aea467dfc47', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f281bf8af484f7a88df7dc8ac06d887', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:10:08.061301 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-7870166a-4f6b-4bed-82ea-c9557eab9bb2 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5" status: 200 len: 1187 microversion: 2.1 time: 0.089686 Oct 30 01:10:08.061925 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 632/1287] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:10:07 2022] GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5 => generated 1187 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:10:09.087366 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-d45d97aa-35db-4302-a24c-47dca7d2594f tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:10:09.087990 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-d45d97aa-35db-4302-a24c-47dca7d2594f tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:10:09.095948 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d45d97aa-35db-4302-a24c-47dca7d2594f tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:09.096441 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d45d97aa-35db-4302-a24c-47dca7d2594f tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:09.096879 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d45d97aa-35db-4302-a24c-47dca7d2594f tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:09.097590 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d45d97aa-35db-4302-a24c-47dca7d2594f tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:09.098043 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d45d97aa-35db-4302-a24c-47dca7d2594f tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:09.098458 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d45d97aa-35db-4302-a24c-47dca7d2594f tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:09.168045 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-d45d97aa-35db-4302-a24c-47dca7d2594f tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baeaad0029f1455497fc3aea467dfc47', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f281bf8af484f7a88df7dc8ac06d887', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:10:09.177929 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-d45d97aa-35db-4302-a24c-47dca7d2594f tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5" status: 200 len: 1205 microversion: 2.1 time: 0.094755 Oct 30 01:10:09.178670 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 656/1288] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:10:09 2022] GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5 => generated 1205 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:10:10.119842 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [req-fc37af06-a586-47ee-852f-1fb2fb947dd7 req-5fe173e4-1960-470a-9a1d-5871533b36ec service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4444bade-70b8-4cef-9037-02cc5773a5d5", "tag": "4025698b-6028-4fe4-ae38-07266f391e05"}]} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:10:10.132965 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-fc37af06-a586-47ee-852f-1fb2fb947dd7 req-5fe173e4-1960-470a-9a1d-5871533b36ec service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:10.133719 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-fc37af06-a586-47ee-852f-1fb2fb947dd7 req-5fe173e4-1960-470a-9a1d-5871533b36ec service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:10.134707 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-fc37af06-a586-47ee-852f-1fb2fb947dd7 req-5fe173e4-1960-470a-9a1d-5871533b36ec service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:10.149863 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.compute.server_external_events [req-fc37af06-a586-47ee-852f-1fb2fb947dd7 req-5fe173e4-1960-470a-9a1d-5871533b36ec service nova] Creating event network-changed:4025698b-6028-4fe4-ae38-07266f391e05 for instance 4444bade-70b8-4cef-9037-02cc5773a5d5 on hv-cinder-maste Oct 30 01:10:10.156131 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [req-fc37af06-a586-47ee-852f-1fb2fb947dd7 req-5fe173e4-1960-470a-9a1d-5871533b36ec service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039420 Oct 30 01:10:10.156787 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 633/1289] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:10:10 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:10:10.201245 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-4c8a2424-2629-411d-837e-16bc0486fe61 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:10:10.201908 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-4c8a2424-2629-411d-837e-16bc0486fe61 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:10:10.209363 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4c8a2424-2629-411d-837e-16bc0486fe61 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:10.209954 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4c8a2424-2629-411d-837e-16bc0486fe61 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:10.210357 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4c8a2424-2629-411d-837e-16bc0486fe61 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:10.211063 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4c8a2424-2629-411d-837e-16bc0486fe61 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:10.211645 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4c8a2424-2629-411d-837e-16bc0486fe61 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:10.212390 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-4c8a2424-2629-411d-837e-16bc0486fe61 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:10.347654 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-4c8a2424-2629-411d-837e-16bc0486fe61 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baeaad0029f1455497fc3aea467dfc47', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f281bf8af484f7a88df7dc8ac06d887', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:10:10.355552 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-4c8a2424-2629-411d-837e-16bc0486fe61 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5" status: 200 len: 1260 microversion: 2.1 time: 0.158304 Oct 30 01:10:10.356165 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 657/1290] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:10:10 2022] GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5 => generated 1260 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:10:11.380358 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-eb8ea230-e2eb-4d9b-b5e6-5551bfc487db tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:10:11.381042 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-eb8ea230-e2eb-4d9b-b5e6-5551bfc487db tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:10:11.387691 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-eb8ea230-e2eb-4d9b-b5e6-5551bfc487db tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:11.388187 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-eb8ea230-e2eb-4d9b-b5e6-5551bfc487db tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:11.388605 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-eb8ea230-e2eb-4d9b-b5e6-5551bfc487db tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:11.389284 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-eb8ea230-e2eb-4d9b-b5e6-5551bfc487db tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:11.389725 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-eb8ea230-e2eb-4d9b-b5e6-5551bfc487db tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:11.390139 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-eb8ea230-e2eb-4d9b-b5e6-5551bfc487db tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:11.531422 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-eb8ea230-e2eb-4d9b-b5e6-5551bfc487db tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baeaad0029f1455497fc3aea467dfc47', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f281bf8af484f7a88df7dc8ac06d887', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:10:11.539537 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-eb8ea230-e2eb-4d9b-b5e6-5551bfc487db tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5" status: 200 len: 1260 microversion: 2.1 time: 0.164101 Oct 30 01:10:11.540128 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 634/1291] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:10:11 2022] GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5 => generated 1260 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:10:12.574308 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-ba5ecc24-98fc-43a8-b651-dc4c8ed82f28 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:10:12.575307 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-ba5ecc24-98fc-43a8-b651-dc4c8ed82f28 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:10:12.583865 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ba5ecc24-98fc-43a8-b651-dc4c8ed82f28 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:12.584372 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ba5ecc24-98fc-43a8-b651-dc4c8ed82f28 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:12.584801 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ba5ecc24-98fc-43a8-b651-dc4c8ed82f28 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:12.585556 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ba5ecc24-98fc-43a8-b651-dc4c8ed82f28 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:12.585991 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ba5ecc24-98fc-43a8-b651-dc4c8ed82f28 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:12.586408 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ba5ecc24-98fc-43a8-b651-dc4c8ed82f28 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:12.720057 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-ba5ecc24-98fc-43a8-b651-dc4c8ed82f28 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baeaad0029f1455497fc3aea467dfc47', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f281bf8af484f7a88df7dc8ac06d887', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:10:12.728045 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-ba5ecc24-98fc-43a8-b651-dc4c8ed82f28 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5" status: 200 len: 1260 microversion: 2.1 time: 0.160692 Oct 30 01:10:12.728595 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 658/1292] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:10:12 2022] GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5 => generated 1260 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:10:13.750976 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-e366c3bd-ccb7-4bd2-9d05-0a4a38e69e79 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:10:13.751592 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-e366c3bd-ccb7-4bd2-9d05-0a4a38e69e79 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:10:13.758895 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e366c3bd-ccb7-4bd2-9d05-0a4a38e69e79 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:13.759374 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e366c3bd-ccb7-4bd2-9d05-0a4a38e69e79 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:13.759833 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e366c3bd-ccb7-4bd2-9d05-0a4a38e69e79 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:13.760519 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e366c3bd-ccb7-4bd2-9d05-0a4a38e69e79 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:13.760963 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e366c3bd-ccb7-4bd2-9d05-0a4a38e69e79 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:13.761373 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e366c3bd-ccb7-4bd2-9d05-0a4a38e69e79 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:13.899484 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-e366c3bd-ccb7-4bd2-9d05-0a4a38e69e79 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baeaad0029f1455497fc3aea467dfc47', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f281bf8af484f7a88df7dc8ac06d887', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:10:13.908171 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-e366c3bd-ccb7-4bd2-9d05-0a4a38e69e79 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5" status: 200 len: 1260 microversion: 2.1 time: 0.161150 Oct 30 01:10:13.908715 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 635/1293] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:10:13 2022] GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5 => generated 1260 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:10:14.934647 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-b5e7a709-1e05-4c90-857c-62dc6349fd9d tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:10:14.935596 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-b5e7a709-1e05-4c90-857c-62dc6349fd9d tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:10:14.943658 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b5e7a709-1e05-4c90-857c-62dc6349fd9d tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:14.944186 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b5e7a709-1e05-4c90-857c-62dc6349fd9d tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:14.944622 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b5e7a709-1e05-4c90-857c-62dc6349fd9d tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:14.945386 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b5e7a709-1e05-4c90-857c-62dc6349fd9d tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:14.945839 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b5e7a709-1e05-4c90-857c-62dc6349fd9d tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:14.946262 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b5e7a709-1e05-4c90-857c-62dc6349fd9d tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:15.091261 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-b5e7a709-1e05-4c90-857c-62dc6349fd9d tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baeaad0029f1455497fc3aea467dfc47', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f281bf8af484f7a88df7dc8ac06d887', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:10:15.103681 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-b5e7a709-1e05-4c90-857c-62dc6349fd9d tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5" status: 200 len: 1260 microversion: 2.1 time: 0.175256 Oct 30 01:10:15.104467 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 659/1294] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:10:14 2022] GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5 => generated 1260 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:10:16.129972 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-3d033b12-068a-4e73-91b7-5d5436018d7e tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:10:16.130662 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-3d033b12-068a-4e73-91b7-5d5436018d7e tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:10:16.138177 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3d033b12-068a-4e73-91b7-5d5436018d7e tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:16.138718 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3d033b12-068a-4e73-91b7-5d5436018d7e tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:16.139197 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3d033b12-068a-4e73-91b7-5d5436018d7e tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:16.140003 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3d033b12-068a-4e73-91b7-5d5436018d7e tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:16.140503 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3d033b12-068a-4e73-91b7-5d5436018d7e tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:16.140967 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3d033b12-068a-4e73-91b7-5d5436018d7e tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:16.279453 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-3d033b12-068a-4e73-91b7-5d5436018d7e tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baeaad0029f1455497fc3aea467dfc47', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f281bf8af484f7a88df7dc8ac06d887', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:10:16.287675 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-3d033b12-068a-4e73-91b7-5d5436018d7e tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5" status: 200 len: 1260 microversion: 2.1 time: 0.163217 Oct 30 01:10:16.288286 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 636/1295] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:10:16 2022] GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5 => generated 1260 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:10:17.310544 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-9cb21c52-d4f1-4615-90ad-39430ea0342b tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:10:17.311159 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-9cb21c52-d4f1-4615-90ad-39430ea0342b tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:10:17.313589 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell0:SELECT=2 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:17.319210 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9cb21c52-d4f1-4615-90ad-39430ea0342b tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:17.319668 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9cb21c52-d4f1-4615-90ad-39430ea0342b tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:17.320122 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9cb21c52-d4f1-4615-90ad-39430ea0342b tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:17.320837 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9cb21c52-d4f1-4615-90ad-39430ea0342b tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:17.321279 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9cb21c52-d4f1-4615-90ad-39430ea0342b tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:17.321692 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9cb21c52-d4f1-4615-90ad-39430ea0342b tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:17.472045 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-9cb21c52-d4f1-4615-90ad-39430ea0342b tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baeaad0029f1455497fc3aea467dfc47', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f281bf8af484f7a88df7dc8ac06d887', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:10:17.483466 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-9cb21c52-d4f1-4615-90ad-39430ea0342b tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5" status: 200 len: 1260 microversion: 2.1 time: 0.176877 Oct 30 01:10:17.484271 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 660/1296] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:10:17 2022] GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5 => generated 1260 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:10:18.511948 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-5229751b-1178-4149-bf02-e7fa0fd584de tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:10:18.513012 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-5229751b-1178-4149-bf02-e7fa0fd584de tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:10:18.516146 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell0:SELECT=2 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:18.517252 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell0:SELECT=6 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:18.529111 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5229751b-1178-4149-bf02-e7fa0fd584de tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:18.529902 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5229751b-1178-4149-bf02-e7fa0fd584de tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:18.530622 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5229751b-1178-4149-bf02-e7fa0fd584de tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:18.531850 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5229751b-1178-4149-bf02-e7fa0fd584de tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:18.532593 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5229751b-1178-4149-bf02-e7fa0fd584de tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:18.533290 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5229751b-1178-4149-bf02-e7fa0fd584de tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:18.678231 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-5229751b-1178-4149-bf02-e7fa0fd584de tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baeaad0029f1455497fc3aea467dfc47', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f281bf8af484f7a88df7dc8ac06d887', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:10:18.686508 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-5229751b-1178-4149-bf02-e7fa0fd584de tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5" status: 200 len: 1260 microversion: 2.1 time: 0.181231 Oct 30 01:10:18.687084 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 637/1297] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:10:18 2022] GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5 => generated 1260 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:10:19.714665 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-7b17ea0d-78fa-46e3-adac-84a13d2a6909 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:10:19.715565 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-7b17ea0d-78fa-46e3-adac-84a13d2a6909 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:10:19.727132 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7b17ea0d-78fa-46e3-adac-84a13d2a6909 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:19.728014 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7b17ea0d-78fa-46e3-adac-84a13d2a6909 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:19.728691 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7b17ea0d-78fa-46e3-adac-84a13d2a6909 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:19.729822 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7b17ea0d-78fa-46e3-adac-84a13d2a6909 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:19.730564 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7b17ea0d-78fa-46e3-adac-84a13d2a6909 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:19.731272 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7b17ea0d-78fa-46e3-adac-84a13d2a6909 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:19.871342 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-7b17ea0d-78fa-46e3-adac-84a13d2a6909 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baeaad0029f1455497fc3aea467dfc47', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f281bf8af484f7a88df7dc8ac06d887', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:10:19.879277 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-7b17ea0d-78fa-46e3-adac-84a13d2a6909 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5" status: 200 len: 1260 microversion: 2.1 time: 0.171476 Oct 30 01:10:19.879880 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 661/1298] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:10:19 2022] GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5 => generated 1260 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:10:20.904411 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-0ba9e90f-3d31-4fab-903a-a14ec006c1bf tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:10:20.905428 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-0ba9e90f-3d31-4fab-903a-a14ec006c1bf tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:10:20.914402 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0ba9e90f-3d31-4fab-903a-a14ec006c1bf tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:20.915051 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0ba9e90f-3d31-4fab-903a-a14ec006c1bf tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:20.915663 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0ba9e90f-3d31-4fab-903a-a14ec006c1bf tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:20.916627 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0ba9e90f-3d31-4fab-903a-a14ec006c1bf tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:20.917228 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0ba9e90f-3d31-4fab-903a-a14ec006c1bf tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:20.917790 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-0ba9e90f-3d31-4fab-903a-a14ec006c1bf tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:21.050429 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-0ba9e90f-3d31-4fab-903a-a14ec006c1bf tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baeaad0029f1455497fc3aea467dfc47', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f281bf8af484f7a88df7dc8ac06d887', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:10:21.058943 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-0ba9e90f-3d31-4fab-903a-a14ec006c1bf tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5" status: 200 len: 1260 microversion: 2.1 time: 0.159844 Oct 30 01:10:21.059511 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 638/1299] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:10:20 2022] GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5 => generated 1260 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:10:22.030561 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [req-d810cb99-5da4-448a-965c-b088a0b439d8 req-adb122ee-b1d5-4d77-9385-3e0ae0188d19 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4444bade-70b8-4cef-9037-02cc5773a5d5", "name": "network-vif-plugged", "status": "completed", "tag": "4025698b-6028-4fe4-ae38-07266f391e05"}]} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:10:22.041461 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-d810cb99-5da4-448a-965c-b088a0b439d8 req-adb122ee-b1d5-4d77-9385-3e0ae0188d19 service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:22.041946 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-d810cb99-5da4-448a-965c-b088a0b439d8 req-adb122ee-b1d5-4d77-9385-3e0ae0188d19 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:22.042379 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-d810cb99-5da4-448a-965c-b088a0b439d8 req-adb122ee-b1d5-4d77-9385-3e0ae0188d19 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:22.053675 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.compute.server_external_events [req-d810cb99-5da4-448a-965c-b088a0b439d8 req-adb122ee-b1d5-4d77-9385-3e0ae0188d19 service nova] Creating event network-vif-plugged:4025698b-6028-4fe4-ae38-07266f391e05 for instance 4444bade-70b8-4cef-9037-02cc5773a5d5 on hv-cinder-maste Oct 30 01:10:22.058779 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [req-d810cb99-5da4-448a-965c-b088a0b439d8 req-adb122ee-b1d5-4d77-9385-3e0ae0188d19 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032054 Oct 30 01:10:22.059351 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 662/1300] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:10:22 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:10:22.089731 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-83072447-ad49-4cc3-83f5-5d4d1a008daf tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:10:22.090455 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-83072447-ad49-4cc3-83f5-5d4d1a008daf tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:10:22.096978 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-83072447-ad49-4cc3-83f5-5d4d1a008daf tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:22.097576 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-83072447-ad49-4cc3-83f5-5d4d1a008daf tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:22.097987 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-83072447-ad49-4cc3-83f5-5d4d1a008daf tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:22.098825 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-83072447-ad49-4cc3-83f5-5d4d1a008daf tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:22.099123 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-83072447-ad49-4cc3-83f5-5d4d1a008daf tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:22.099554 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-83072447-ad49-4cc3-83f5-5d4d1a008daf tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:22.237047 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-83072447-ad49-4cc3-83f5-5d4d1a008daf tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baeaad0029f1455497fc3aea467dfc47', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f281bf8af484f7a88df7dc8ac06d887', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:10:22.245518 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-83072447-ad49-4cc3-83f5-5d4d1a008daf tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5" status: 200 len: 1260 microversion: 2.1 time: 0.159681 Oct 30 01:10:22.246163 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 639/1301] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:10:22 2022] GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5 => generated 1260 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:10:23.270926 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-3cc0ef56-211c-4249-9351-66bcfa93ed83 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:10:23.271639 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-3cc0ef56-211c-4249-9351-66bcfa93ed83 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:10:23.278454 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3cc0ef56-211c-4249-9351-66bcfa93ed83 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:23.278965 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3cc0ef56-211c-4249-9351-66bcfa93ed83 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:23.279402 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3cc0ef56-211c-4249-9351-66bcfa93ed83 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:23.280132 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3cc0ef56-211c-4249-9351-66bcfa93ed83 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:23.280579 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3cc0ef56-211c-4249-9351-66bcfa93ed83 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:23.280998 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-3cc0ef56-211c-4249-9351-66bcfa93ed83 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:23.421111 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-3cc0ef56-211c-4249-9351-66bcfa93ed83 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baeaad0029f1455497fc3aea467dfc47', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f281bf8af484f7a88df7dc8ac06d887', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:10:23.429322 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-3cc0ef56-211c-4249-9351-66bcfa93ed83 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5" status: 200 len: 1260 microversion: 2.1 time: 0.163761 Oct 30 01:10:23.429877 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 663/1302] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:10:23 2022] GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5 => generated 1260 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:10:24.451505 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-23f6b8a4-ba0e-47ea-a509-05b70edbdba1 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:10:24.452139 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-23f6b8a4-ba0e-47ea-a509-05b70edbdba1 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:10:24.459223 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-23f6b8a4-ba0e-47ea-a509-05b70edbdba1 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:24.459704 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-23f6b8a4-ba0e-47ea-a509-05b70edbdba1 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:24.460159 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-23f6b8a4-ba0e-47ea-a509-05b70edbdba1 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:24.460851 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-23f6b8a4-ba0e-47ea-a509-05b70edbdba1 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:24.461297 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-23f6b8a4-ba0e-47ea-a509-05b70edbdba1 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:24.461714 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-23f6b8a4-ba0e-47ea-a509-05b70edbdba1 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:24.602171 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-23f6b8a4-ba0e-47ea-a509-05b70edbdba1 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baeaad0029f1455497fc3aea467dfc47', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f281bf8af484f7a88df7dc8ac06d887', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:10:24.613416 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-23f6b8a4-ba0e-47ea-a509-05b70edbdba1 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5" status: 200 len: 1445 microversion: 2.1 time: 0.165681 Oct 30 01:10:24.614201 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 640/1303] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:10:24 2022] GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5 => generated 1445 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:10:24.636534 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-d3d3f85f-9447-42d3-9c33-83422c9e4614 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:10:24.637147 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-d3d3f85f-9447-42d3-9c33-83422c9e4614 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:10:24.643856 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d3d3f85f-9447-42d3-9c33-83422c9e4614 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:24.644429 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d3d3f85f-9447-42d3-9c33-83422c9e4614 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:24.644871 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d3d3f85f-9447-42d3-9c33-83422c9e4614 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:24.645591 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d3d3f85f-9447-42d3-9c33-83422c9e4614 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:24.646071 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d3d3f85f-9447-42d3-9c33-83422c9e4614 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:24.646500 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d3d3f85f-9447-42d3-9c33-83422c9e4614 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:24.796365 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-d3d3f85f-9447-42d3-9c33-83422c9e4614 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baeaad0029f1455497fc3aea467dfc47', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f281bf8af484f7a88df7dc8ac06d887', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:10:24.804920 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-d3d3f85f-9447-42d3-9c33-83422c9e4614 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5" status: 200 len: 1445 microversion: 2.1 time: 0.172367 Oct 30 01:10:24.805499 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 664/1304] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:10:24 2022] GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5 => generated 1445 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:10:26.625098 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [req-7a77aaa2-14c7-476a-b9c5-f7b1c38d94f3 req-96658e56-003d-4bee-b1a8-89cea07b6557 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4444bade-70b8-4cef-9037-02cc5773a5d5", "tag": "4025698b-6028-4fe4-ae38-07266f391e05"}]} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:10:26.634916 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-7a77aaa2-14c7-476a-b9c5-f7b1c38d94f3 req-96658e56-003d-4bee-b1a8-89cea07b6557 service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:26.635417 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-7a77aaa2-14c7-476a-b9c5-f7b1c38d94f3 req-96658e56-003d-4bee-b1a8-89cea07b6557 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:26.635876 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-7a77aaa2-14c7-476a-b9c5-f7b1c38d94f3 req-96658e56-003d-4bee-b1a8-89cea07b6557 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:26.647024 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.compute.server_external_events [req-7a77aaa2-14c7-476a-b9c5-f7b1c38d94f3 req-96658e56-003d-4bee-b1a8-89cea07b6557 service nova] Creating event network-changed:4025698b-6028-4fe4-ae38-07266f391e05 for instance 4444bade-70b8-4cef-9037-02cc5773a5d5 on hv-cinder-maste Oct 30 01:10:26.652334 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [req-7a77aaa2-14c7-476a-b9c5-f7b1c38d94f3 req-96658e56-003d-4bee-b1a8-89cea07b6557 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.090935 Oct 30 01:10:26.653160 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 641/1305] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:10:26 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:10:39.371709 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-5b12361e-034a-4e9f-ad83-fbc8b3379f78 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:10:39.372392 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-5b12361e-034a-4e9f-ad83-fbc8b3379f78 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: 91a8549d-4f75-4c98-9773-98efe4196889] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:10:39.375579 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_api:SELECT=22 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:39.376434 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell1:SELECT=50 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:39.383367 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5b12361e-034a-4e9f-ad83-fbc8b3379f78 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:39.383937 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5b12361e-034a-4e9f-ad83-fbc8b3379f78 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:39.384433 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5b12361e-034a-4e9f-ad83-fbc8b3379f78 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:39.385166 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5b12361e-034a-4e9f-ad83-fbc8b3379f78 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:39.385619 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5b12361e-034a-4e9f-ad83-fbc8b3379f78 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:39.386057 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5b12361e-034a-4e9f-ad83-fbc8b3379f78 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:39.411842 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-5b12361e-034a-4e9f-ad83-fbc8b3379f78 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: 91a8549d-4f75-4c98-9773-98efe4196889] Going to try to terminate instance {{(pid=118961) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 30 01:10:39.499621 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-5b12361e-034a-4e9f-ad83-fbc8b3379f78 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "DELETE /compute/v2.1/servers/91a8549d-4f75-4c98-9773-98efe4196889" status: 204 len: 0 microversion: 2.1 time: 0.132879 Oct 30 01:10:39.500248 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 665/1306] 10.222.0.16 () {58 vars in 1239 bytes} [Sun Oct 30 01:10:39 2022] DELETE /compute/v2.1/servers/91a8549d-4f75-4c98-9773-98efe4196889 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 30 01:10:39.507350 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_api:SELECT=35,nova_api:INSERT=3 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:39.509101 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell1:SELECT=54 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:39.513953 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-3ad85421-268f-4682-ad93-ac51d8691ae6 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:10:39.514657 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-3ad85421-268f-4682-ad93-ac51d8691ae6 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: 91a8549d-4f75-4c98-9773-98efe4196889] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:10:39.523475 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3ad85421-268f-4682-ad93-ac51d8691ae6 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:39.523997 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3ad85421-268f-4682-ad93-ac51d8691ae6 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:39.524432 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3ad85421-268f-4682-ad93-ac51d8691ae6 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:39.525120 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3ad85421-268f-4682-ad93-ac51d8691ae6 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:39.525569 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3ad85421-268f-4682-ad93-ac51d8691ae6 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:39.525989 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3ad85421-268f-4682-ad93-ac51d8691ae6 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:39.667342 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-3ad85421-268f-4682-ad93-ac51d8691ae6 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:10:39.675534 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-3ad85421-268f-4682-ad93-ac51d8691ae6 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/91a8549d-4f75-4c98-9773-98efe4196889" status: 200 len: 1578 microversion: 2.1 time: 0.170410 Oct 30 01:10:39.676257 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 642/1307] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:10:39 2022] GET /compute/v2.1/servers/91a8549d-4f75-4c98-9773-98efe4196889 => generated 1578 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:10:40.703931 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-1ca08390-8eed-41a5-b479-c6901df2d72c tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:10:40.704945 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-1ca08390-8eed-41a5-b479-c6901df2d72c tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: 91a8549d-4f75-4c98-9773-98efe4196889] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:10:40.713031 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1ca08390-8eed-41a5-b479-c6901df2d72c tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:40.713527 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1ca08390-8eed-41a5-b479-c6901df2d72c tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:40.713972 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1ca08390-8eed-41a5-b479-c6901df2d72c tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:40.714693 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1ca08390-8eed-41a5-b479-c6901df2d72c tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:40.715150 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1ca08390-8eed-41a5-b479-c6901df2d72c tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:40.715570 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1ca08390-8eed-41a5-b479-c6901df2d72c tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:40.849773 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-1ca08390-8eed-41a5-b479-c6901df2d72c tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:10:40.857917 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-1ca08390-8eed-41a5-b479-c6901df2d72c tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/91a8549d-4f75-4c98-9773-98efe4196889" status: 200 len: 1578 microversion: 2.1 time: 0.161340 Oct 30 01:10:40.858494 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 666/1308] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:10:40 2022] GET /compute/v2.1/servers/91a8549d-4f75-4c98-9773-98efe4196889 => generated 1578 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:10:41.881832 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-dbfe63b1-0ab0-430a-8b8f-2350e67c1f94 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:10:41.882428 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-dbfe63b1-0ab0-430a-8b8f-2350e67c1f94 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: 91a8549d-4f75-4c98-9773-98efe4196889] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:10:41.889354 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dbfe63b1-0ab0-430a-8b8f-2350e67c1f94 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:41.889832 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dbfe63b1-0ab0-430a-8b8f-2350e67c1f94 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:41.890254 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dbfe63b1-0ab0-430a-8b8f-2350e67c1f94 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:41.890956 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dbfe63b1-0ab0-430a-8b8f-2350e67c1f94 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:41.891445 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dbfe63b1-0ab0-430a-8b8f-2350e67c1f94 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:41.891831 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-dbfe63b1-0ab0-430a-8b8f-2350e67c1f94 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:42.026431 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-dbfe63b1-0ab0-430a-8b8f-2350e67c1f94 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:10:42.034486 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-dbfe63b1-0ab0-430a-8b8f-2350e67c1f94 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/91a8549d-4f75-4c98-9773-98efe4196889" status: 200 len: 1578 microversion: 2.1 time: 0.157286 Oct 30 01:10:42.035259 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 643/1309] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:10:41 2022] GET /compute/v2.1/servers/91a8549d-4f75-4c98-9773-98efe4196889 => generated 1578 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:10:42.335097 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [req-aff87f96-a8db-4984-be02-8d86f0e3b984 req-416709a2-72cd-45b5-ac29-4f42bc850830 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "91a8549d-4f75-4c98-9773-98efe4196889", "name": "network-vif-deleted", "tag": "7cefd299-3b60-4011-b5ad-6ad6926ac49d"}]} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:10:42.349134 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-aff87f96-a8db-4984-be02-8d86f0e3b984 req-416709a2-72cd-45b5-ac29-4f42bc850830 service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:42.349768 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-aff87f96-a8db-4984-be02-8d86f0e3b984 req-416709a2-72cd-45b5-ac29-4f42bc850830 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:42.350112 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-aff87f96-a8db-4984-be02-8d86f0e3b984 req-416709a2-72cd-45b5-ac29-4f42bc850830 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:42.364849 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.compute.server_external_events [req-aff87f96-a8db-4984-be02-8d86f0e3b984 req-416709a2-72cd-45b5-ac29-4f42bc850830 service nova] Creating event network-vif-deleted:7cefd299-3b60-4011-b5ad-6ad6926ac49d for instance 91a8549d-4f75-4c98-9773-98efe4196889 on hv-cinder-maste Oct 30 01:10:42.370723 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [req-aff87f96-a8db-4984-be02-8d86f0e3b984 req-416709a2-72cd-45b5-ac29-4f42bc850830 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039631 Oct 30 01:10:42.371373 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 667/1310] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:10:42 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:10:43.056986 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-ac14fcae-78de-4cda-b872-fc60db0c5162 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:10:43.057673 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-ac14fcae-78de-4cda-b872-fc60db0c5162 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: 91a8549d-4f75-4c98-9773-98efe4196889] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:10:43.064658 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ac14fcae-78de-4cda-b872-fc60db0c5162 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:43.065255 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ac14fcae-78de-4cda-b872-fc60db0c5162 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:43.065752 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ac14fcae-78de-4cda-b872-fc60db0c5162 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:43.066449 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ac14fcae-78de-4cda-b872-fc60db0c5162 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:43.066897 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ac14fcae-78de-4cda-b872-fc60db0c5162 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:43.067316 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ac14fcae-78de-4cda-b872-fc60db0c5162 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:43.100013 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ac14fcae-78de-4cda-b872-fc60db0c5162 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:43.100600 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ac14fcae-78de-4cda-b872-fc60db0c5162 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:43.101048 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ac14fcae-78de-4cda-b872-fc60db0c5162 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:43.101594 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.objects.instance [None req-ac14fcae-78de-4cda-b872-fc60db0c5162 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lazy-loading 'fault' on Instance uuid 91a8549d-4f75-4c98-9773-98efe4196889 {{(pid=118962) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 30 01:10:43.146020 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-ac14fcae-78de-4cda-b872-fc60db0c5162 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:10:43.155147 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-ac14fcae-78de-4cda-b872-fc60db0c5162 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/91a8549d-4f75-4c98-9773-98efe4196889" status: 200 len: 1229 microversion: 2.1 time: 0.102147 Oct 30 01:10:43.155774 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 644/1311] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:10:43 2022] GET /compute/v2.1/servers/91a8549d-4f75-4c98-9773-98efe4196889 => generated 1229 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:10:44.184723 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-6db48abd-26a4-4c03-b0aa-7552d4be1661 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:10:44.185742 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-6db48abd-26a4-4c03-b0aa-7552d4be1661 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: 91a8549d-4f75-4c98-9773-98efe4196889] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:10:44.196372 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6db48abd-26a4-4c03-b0aa-7552d4be1661 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:44.197166 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6db48abd-26a4-4c03-b0aa-7552d4be1661 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:44.197892 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6db48abd-26a4-4c03-b0aa-7552d4be1661 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:44.199031 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6db48abd-26a4-4c03-b0aa-7552d4be1661 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:44.199878 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6db48abd-26a4-4c03-b0aa-7552d4be1661 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:44.200517 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6db48abd-26a4-4c03-b0aa-7552d4be1661 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:44.220213 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.wsgi [None req-6db48abd-26a4-4c03-b0aa-7552d4be1661 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] HTTP exception thrown: Instance 91a8549d-4f75-4c98-9773-98efe4196889 could not be found. Oct 30 01:10:44.221881 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-6db48abd-26a4-4c03-b0aa-7552d4be1661 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Request method failure captured: Oct 30 01:10:44.221881 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: request: GET /compute/v2.1/servers/91a8549d-4f75-4c98-9773-98efe4196889 HTTP/1.1 Oct 30 01:10:44.221881 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Accept: application/json Oct 30 01:10:44.221881 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Accept-Encoding: identity Oct 30 01:10:44.221881 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Connection: close Oct 30 01:10:44.221881 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Content-Length: 0 Oct 30 01:10:44.221881 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Content-Type: application/json Oct 30 01:10:44.221881 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Host: 10.222.0.16 Oct 30 01:10:44.221881 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Openstack-System-Scope: None Oct 30 01:10:44.221881 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: User-Agent: python-urllib3/1.26.12 Oct 30 01:10:44.221881 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Auth-Token: gAAAAABjXc5Cle3iH9ItI2HXvwQRx6PZePcxrmkQWKXntioRsEcG8bGDbeJjw9hikhUg8ep4tjFQcKripNAxSYrAA9oY711XA708rHY3-1IRhoTotW6hXRFu4pB5gkihg7xD1caEs6KUEMpV4RpQtad_eiHzmSTNbKnNiaGaXF6jwXfx9XC17J0 Oct 30 01:10:44.221881 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Domain-Id: None Oct 30 01:10:44.221881 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Domain-Name: None Oct 30 01:10:44.221881 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Identity-Status: Confirmed Oct 30 01:10:44.221881 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Is-Admin-Project: True Oct 30 01:10:44.221881 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Domain-Id: default Oct 30 01:10:44.221881 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Domain-Name: Default Oct 30 01:10:44.221881 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Id: 238d71be4e094cb6b84f9a48a8a2c79c Oct 30 01:10:44.221881 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Name: tempest-TestVolumeBootPattern-653595049 Oct 30 01:10:44.221881 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Role: reader,member Oct 30 01:10:44.221881 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Roles: reader,member Oct 30 01:10:44.223711 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_238d71be4e094cb6b84f9a48a8a2c79c"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/238d71be4e094cb6b84f9a48a8a2c79c"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/238d71be4e094cb6b84f9a48a8a2c79c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/238d71be4e094cb6b84f9a48a8a2c79c"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 01:10:44.223711 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant: tempest-TestVolumeBootPattern-653595049 Oct 30 01:10:44.223711 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant-Id: 238d71be4e094cb6b84f9a48a8a2c79c Oct 30 01:10:44.223711 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant-Name: tempest-TestVolumeBootPattern-653595049 Oct 30 01:10:44.223711 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User: tempest-TestVolumeBootPattern-653595049-project Oct 30 01:10:44.223711 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Domain-Id: default Oct 30 01:10:44.223711 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Domain-Name: Default Oct 30 01:10:44.223711 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Id: 96e0cd00e4ef4a2da960aa5d4f66972a Oct 30 01:10:44.223711 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Name: tempest-TestVolumeBootPattern-653595049-project Oct 30 01:10:44.223711 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: method: > Oct 30 01:10:44.223711 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: exception: Instance 91a8549d-4f75-4c98-9773-98efe4196889 could not be found. Oct 30 01:10:44.223711 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: action_args: {'id': '91a8549d-4f75-4c98-9773-98efe4196889'} Oct 30 01:10:44.224818 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 01:10:44.224818 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:10:44.224818 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 30 01:10:44.224818 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 30 01:10:44.224818 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 30 01:10:44.224818 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 30 01:10:44.224818 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 30 01:10:44.224818 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 30 01:10:44.224818 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 30 01:10:44.224818 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 30 01:10:44.224818 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 91a8549d-4f75-4c98-9773-98efe4196889 could not be found. Oct 30 01:10:44.224818 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:10:44.224818 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:10:44.224818 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:10:44.224818 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:10:44.224818 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:10:44.224818 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:10:44.224818 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 01:10:44.224818 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 01:10:44.224818 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 01:10:44.226713 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 01:10:44.226713 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 30 01:10:44.226713 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 30 01:10:44.226713 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 30 01:10:44.226713 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 30 01:10:44.226713 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 30 01:10:44.226713 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 30 01:10:44.226713 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 91a8549d-4f75-4c98-9773-98efe4196889 could not be found. Oct 30 01:10:44.226713 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:10:44.226713 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:10:44.226713 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:10:44.226713 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:10:44.226713 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:10:44.226713 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:10:44.226713 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 01:10:44.226713 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 01:10:44.226713 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 91a8549d-4f75-4c98-9773-98efe4196889 could not be found. Oct 30 01:10:44.226713 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:10:44.228298 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-6db48abd-26a4-4c03-b0aa-7552d4be1661 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Returning 404 to user: Instance 91a8549d-4f75-4c98-9773-98efe4196889 could not be found. {{(pid=118961) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 01:10:44.228298 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-6db48abd-26a4-4c03-b0aa-7552d4be1661 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/91a8549d-4f75-4c98-9773-98efe4196889" status: 404 len: 111 microversion: 2.1 time: 0.046871 Oct 30 01:10:44.228298 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 668/1312] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:10:44 2022] GET /compute/v2.1/servers/91a8549d-4f75-4c98-9773-98efe4196889 => generated 111 bytes in 49 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 30 01:10:44.324152 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-ecc330e0-0b90-4fc8-afeb-460a14144383 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-819333328", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1884997751", "security_groups": [{"name": "tempest-secgroup-smoke-1108501088"}], "block_device_mapping_v2": [{"uuid": "54171fe1-01cf-43fe-9e0f-62297f055415", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "c1a88451-f287-4e35-b7e8-a6d4e4aaa603"}]}} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:10:44.442002 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.network.neutron [None req-ecc330e0-0b90-4fc8-afeb-460a14144383 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] validate_networks() for [('c1a88451-f287-4e35-b7e8-a6d4e4aaa603', None, None, None, None, None)] {{(pid=118962) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 30 01:10:44.810795 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.virt.hardware [None req-ecc330e0-0b90-4fc8-afeb-460a14144383 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Flavor limits 0:0:0 {{(pid=118962) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 30 01:10:44.811217 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.virt.hardware [None req-ecc330e0-0b90-4fc8-afeb-460a14144383 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Image limits 0:0:0 {{(pid=118962) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 30 01:10:44.811685 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.virt.hardware [None req-ecc330e0-0b90-4fc8-afeb-460a14144383 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Flavor pref 0:0:0 {{(pid=118962) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 30 01:10:44.812134 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.virt.hardware [None req-ecc330e0-0b90-4fc8-afeb-460a14144383 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Image pref 0:0:0 {{(pid=118962) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 30 01:10:44.812538 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.virt.hardware [None req-ecc330e0-0b90-4fc8-afeb-460a14144383 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118962) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 30 01:10:44.822005 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ecc330e0-0b90-4fc8-afeb-460a14144383 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:44.822489 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ecc330e0-0b90-4fc8-afeb-460a14144383 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:44.822923 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ecc330e0-0b90-4fc8-afeb-460a14144383 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:44.823659 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ecc330e0-0b90-4fc8-afeb-460a14144383 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:44.824142 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ecc330e0-0b90-4fc8-afeb-460a14144383 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:44.824559 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-ecc330e0-0b90-4fc8-afeb-460a14144383 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:44.841813 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.quota [None req-ecc330e0-0b90-4fc8-afeb-460a14144383 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Getting quotas for project 238d71be4e094cb6b84f9a48a8a2c79c. Resources: {'ram', 'instances', 'cores'} {{(pid=118962) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 30 01:10:44.845837 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.quota [None req-ecc330e0-0b90-4fc8-afeb-460a14144383 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Getting quotas for user 96e0cd00e4ef4a2da960aa5d4f66972a and project 238d71be4e094cb6b84f9a48a8a2c79c. Resources: {'ram', 'instances', 'cores'} {{(pid=118962) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 30 01:10:44.907698 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-ecc330e0-0b90-4fc8-afeb-460a14144383 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Going to run 1 instances... {{(pid=118962) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 30 01:10:44.909857 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-ecc330e0-0b90-4fc8-afeb-460a14144383 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: 80d461e9-eac2-4d78-9e78-d37723f1543b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='54171fe1-01cf-43fe-9e0f-62297f055415',volume_size=None,volume_type=None)] {{(pid=118962) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 30 01:10:45.056729 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-ecc330e0-0b90-4fc8-afeb-460a14144383 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.736547 Oct 30 01:10:45.057375 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 645/1313] 10.222.0.16 () {60 vars in 1147 bytes} [Sun Oct 30 01:10:44 2022] POST /compute/v2.1/servers => generated 406 bytes in 738 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 30 01:10:45.068796 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-91f97a42-3741-4426-a9b7-d2595277876f tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:10:45.069451 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-91f97a42-3741-4426-a9b7-d2595277876f tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: 80d461e9-eac2-4d78-9e78-d37723f1543b] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:10:45.124344 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-91f97a42-3741-4426-a9b7-d2595277876f tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:10:45.131594 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-91f97a42-3741-4426-a9b7-d2595277876f tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/80d461e9-eac2-4d78-9e78-d37723f1543b" status: 200 len: 1094 microversion: 2.1 time: 0.066772 Oct 30 01:10:45.132193 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 669/1314] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:10:45 2022] GET /compute/v2.1/servers/80d461e9-eac2-4d78-9e78-d37723f1543b => generated 1094 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:10:46.155441 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-ea837df7-8051-42f6-8917-e79638e42490 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:10:46.156081 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-ea837df7-8051-42f6-8917-e79638e42490 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: 80d461e9-eac2-4d78-9e78-d37723f1543b] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:10:46.212807 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-ea837df7-8051-42f6-8917-e79638e42490 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:10:46.220360 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-ea837df7-8051-42f6-8917-e79638e42490 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/80d461e9-eac2-4d78-9e78-d37723f1543b" status: 200 len: 1094 microversion: 2.1 time: 0.070015 Oct 30 01:10:46.220929 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 646/1315] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:10:46 2022] GET /compute/v2.1/servers/80d461e9-eac2-4d78-9e78-d37723f1543b => generated 1094 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:10:47.243606 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-ad524d40-e93c-4221-9932-00422656e8ef tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:10:47.244277 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-ad524d40-e93c-4221-9932-00422656e8ef tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: 80d461e9-eac2-4d78-9e78-d37723f1543b] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:10:47.580896 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-ad524d40-e93c-4221-9932-00422656e8ef tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:10:47.592869 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-ad524d40-e93c-4221-9932-00422656e8ef tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/80d461e9-eac2-4d78-9e78-d37723f1543b" status: 200 len: 1094 microversion: 2.1 time: 0.352764 Oct 30 01:10:47.593769 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 670/1316] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:10:47 2022] GET /compute/v2.1/servers/80d461e9-eac2-4d78-9e78-d37723f1543b => generated 1094 bytes in 355 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:10:48.620158 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-cc473442-720b-47a1-879a-8fcd81ab515e tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:10:48.620690 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-cc473442-720b-47a1-879a-8fcd81ab515e tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: 80d461e9-eac2-4d78-9e78-d37723f1543b] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:10:48.630166 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-cc473442-720b-47a1-879a-8fcd81ab515e tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:48.630642 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-cc473442-720b-47a1-879a-8fcd81ab515e tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:48.631067 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-cc473442-720b-47a1-879a-8fcd81ab515e tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:48.631800 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-cc473442-720b-47a1-879a-8fcd81ab515e tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:48.632245 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-cc473442-720b-47a1-879a-8fcd81ab515e tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:48.632662 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-cc473442-720b-47a1-879a-8fcd81ab515e tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:48.692538 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-cc473442-720b-47a1-879a-8fcd81ab515e tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:10:48.700938 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-cc473442-720b-47a1-879a-8fcd81ab515e tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/80d461e9-eac2-4d78-9e78-d37723f1543b" status: 200 len: 1144 microversion: 2.1 time: 0.085487 Oct 30 01:10:48.701516 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 647/1317] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:10:48 2022] GET /compute/v2.1/servers/80d461e9-eac2-4d78-9e78-d37723f1543b => generated 1144 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:10:49.724256 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-5a2b7ec4-eaae-4715-a716-eecbb6d1cd76 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:10:49.724841 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-5a2b7ec4-eaae-4715-a716-eecbb6d1cd76 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: 80d461e9-eac2-4d78-9e78-d37723f1543b] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:10:49.731935 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5a2b7ec4-eaae-4715-a716-eecbb6d1cd76 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:49.732420 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5a2b7ec4-eaae-4715-a716-eecbb6d1cd76 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:49.732857 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5a2b7ec4-eaae-4715-a716-eecbb6d1cd76 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:49.733546 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5a2b7ec4-eaae-4715-a716-eecbb6d1cd76 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:49.733992 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5a2b7ec4-eaae-4715-a716-eecbb6d1cd76 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:49.734410 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5a2b7ec4-eaae-4715-a716-eecbb6d1cd76 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:49.796983 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-5a2b7ec4-eaae-4715-a716-eecbb6d1cd76 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:10:49.805112 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-5a2b7ec4-eaae-4715-a716-eecbb6d1cd76 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/80d461e9-eac2-4d78-9e78-d37723f1543b" status: 200 len: 1136 microversion: 2.1 time: 0.084797 Oct 30 01:10:49.805679 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 671/1318] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:10:49 2022] GET /compute/v2.1/servers/80d461e9-eac2-4d78-9e78-d37723f1543b => generated 1136 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:10:50.829527 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-5930baaa-73be-466d-86f1-a721a1dc2e56 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:10:50.830820 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-5930baaa-73be-466d-86f1-a721a1dc2e56 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: 80d461e9-eac2-4d78-9e78-d37723f1543b] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:10:50.837895 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5930baaa-73be-466d-86f1-a721a1dc2e56 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:50.838408 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5930baaa-73be-466d-86f1-a721a1dc2e56 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:50.838843 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5930baaa-73be-466d-86f1-a721a1dc2e56 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:50.839534 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5930baaa-73be-466d-86f1-a721a1dc2e56 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:50.840006 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5930baaa-73be-466d-86f1-a721a1dc2e56 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:50.840419 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-5930baaa-73be-466d-86f1-a721a1dc2e56 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:50.902103 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-5930baaa-73be-466d-86f1-a721a1dc2e56 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:10:50.909855 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-5930baaa-73be-466d-86f1-a721a1dc2e56 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/80d461e9-eac2-4d78-9e78-d37723f1543b" status: 200 len: 1210 microversion: 2.1 time: 0.084291 Oct 30 01:10:50.910425 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 648/1319] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:10:50 2022] GET /compute/v2.1/servers/80d461e9-eac2-4d78-9e78-d37723f1543b => generated 1210 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:10:51.932904 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-371fb5fc-1f72-403b-b65e-aea7db36f0f8 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:10:51.933822 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-371fb5fc-1f72-403b-b65e-aea7db36f0f8 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: 80d461e9-eac2-4d78-9e78-d37723f1543b] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:10:51.940594 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-371fb5fc-1f72-403b-b65e-aea7db36f0f8 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:51.940996 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-371fb5fc-1f72-403b-b65e-aea7db36f0f8 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:51.941431 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-371fb5fc-1f72-403b-b65e-aea7db36f0f8 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:51.942132 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-371fb5fc-1f72-403b-b65e-aea7db36f0f8 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:51.942581 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-371fb5fc-1f72-403b-b65e-aea7db36f0f8 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:51.943001 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-371fb5fc-1f72-403b-b65e-aea7db36f0f8 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:52.046232 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [req-10604dc6-7b24-46cd-a9bc-aa23dd73e15c req-467a0306-8995-49d9-afe7-914fe9f29ccc service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "80d461e9-eac2-4d78-9e78-d37723f1543b", "tag": "49a23065-cebc-4735-bcae-f52ca512b794"}]} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:10:52.063620 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-10604dc6-7b24-46cd-a9bc-aa23dd73e15c req-467a0306-8995-49d9-afe7-914fe9f29ccc service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:52.064115 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-10604dc6-7b24-46cd-a9bc-aa23dd73e15c req-467a0306-8995-49d9-afe7-914fe9f29ccc service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:52.064691 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-10604dc6-7b24-46cd-a9bc-aa23dd73e15c req-467a0306-8995-49d9-afe7-914fe9f29ccc service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:52.076932 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.compute.server_external_events [req-10604dc6-7b24-46cd-a9bc-aa23dd73e15c req-467a0306-8995-49d9-afe7-914fe9f29ccc service nova] Creating event network-changed:49a23065-cebc-4735-bcae-f52ca512b794 for instance 80d461e9-eac2-4d78-9e78-d37723f1543b on hv-cinder-maste Oct 30 01:10:52.083366 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [req-10604dc6-7b24-46cd-a9bc-aa23dd73e15c req-467a0306-8995-49d9-afe7-914fe9f29ccc service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.045245 Oct 30 01:10:52.084725 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 649/1320] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:10:52 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:10:52.096496 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-371fb5fc-1f72-403b-b65e-aea7db36f0f8 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:10:52.106601 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-371fb5fc-1f72-403b-b65e-aea7db36f0f8 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/80d461e9-eac2-4d78-9e78-d37723f1543b" status: 200 len: 1266 microversion: 2.1 time: 0.177505 Oct 30 01:10:52.107258 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 672/1321] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:10:51 2022] GET /compute/v2.1/servers/80d461e9-eac2-4d78-9e78-d37723f1543b => generated 1266 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:10:53.135167 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-735f40dc-35e0-4a5a-921f-0c49f7e0a49c tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:10:53.136072 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-735f40dc-35e0-4a5a-921f-0c49f7e0a49c tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: 80d461e9-eac2-4d78-9e78-d37723f1543b] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:10:53.144429 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-735f40dc-35e0-4a5a-921f-0c49f7e0a49c tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:53.145030 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-735f40dc-35e0-4a5a-921f-0c49f7e0a49c tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:53.145576 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-735f40dc-35e0-4a5a-921f-0c49f7e0a49c tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:53.146434 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-735f40dc-35e0-4a5a-921f-0c49f7e0a49c tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:53.146987 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-735f40dc-35e0-4a5a-921f-0c49f7e0a49c tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:53.147538 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-735f40dc-35e0-4a5a-921f-0c49f7e0a49c tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:53.340258 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-735f40dc-35e0-4a5a-921f-0c49f7e0a49c tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:10:53.349420 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-735f40dc-35e0-4a5a-921f-0c49f7e0a49c tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/80d461e9-eac2-4d78-9e78-d37723f1543b" status: 200 len: 1266 microversion: 2.1 time: 0.219843 Oct 30 01:10:53.350629 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 650/1322] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:10:53 2022] GET /compute/v2.1/servers/80d461e9-eac2-4d78-9e78-d37723f1543b => generated 1266 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:10:54.377969 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-92403573-bce6-4968-ac99-44675bf6b7b2 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:10:54.378720 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-92403573-bce6-4968-ac99-44675bf6b7b2 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: 80d461e9-eac2-4d78-9e78-d37723f1543b] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:10:54.385816 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-92403573-bce6-4968-ac99-44675bf6b7b2 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:54.386300 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-92403573-bce6-4968-ac99-44675bf6b7b2 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:54.386739 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-92403573-bce6-4968-ac99-44675bf6b7b2 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:54.387429 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-92403573-bce6-4968-ac99-44675bf6b7b2 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:54.387912 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-92403573-bce6-4968-ac99-44675bf6b7b2 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:54.388331 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-92403573-bce6-4968-ac99-44675bf6b7b2 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:54.531679 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-92403573-bce6-4968-ac99-44675bf6b7b2 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:10:54.544666 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-92403573-bce6-4968-ac99-44675bf6b7b2 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/80d461e9-eac2-4d78-9e78-d37723f1543b" status: 200 len: 1266 microversion: 2.1 time: 0.170923 Oct 30 01:10:54.545577 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 673/1323] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:10:54 2022] GET /compute/v2.1/servers/80d461e9-eac2-4d78-9e78-d37723f1543b => generated 1266 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:10:55.573784 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-23b73fbb-4246-4bbd-9534-657fe5e4598a tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:10:55.574839 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-23b73fbb-4246-4bbd-9534-657fe5e4598a tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: 80d461e9-eac2-4d78-9e78-d37723f1543b] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:10:55.578613 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell0:SELECT=2 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:55.586871 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-23b73fbb-4246-4bbd-9534-657fe5e4598a tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:55.587346 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-23b73fbb-4246-4bbd-9534-657fe5e4598a tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:55.587769 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-23b73fbb-4246-4bbd-9534-657fe5e4598a tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:55.588472 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-23b73fbb-4246-4bbd-9534-657fe5e4598a tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:55.588912 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-23b73fbb-4246-4bbd-9534-657fe5e4598a tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:55.589323 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-23b73fbb-4246-4bbd-9534-657fe5e4598a tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:55.735103 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-23b73fbb-4246-4bbd-9534-657fe5e4598a tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:10:55.743331 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-23b73fbb-4246-4bbd-9534-657fe5e4598a tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/80d461e9-eac2-4d78-9e78-d37723f1543b" status: 200 len: 1266 microversion: 2.1 time: 0.176072 Oct 30 01:10:55.743934 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 651/1324] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:10:55 2022] GET /compute/v2.1/servers/80d461e9-eac2-4d78-9e78-d37723f1543b => generated 1266 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:10:56.766467 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-077bc7bd-6dd2-43cf-b80b-7966c7000762 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:10:56.767096 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-077bc7bd-6dd2-43cf-b80b-7966c7000762 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: 80d461e9-eac2-4d78-9e78-d37723f1543b] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:10:56.774033 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-077bc7bd-6dd2-43cf-b80b-7966c7000762 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:56.774527 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-077bc7bd-6dd2-43cf-b80b-7966c7000762 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:56.774966 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-077bc7bd-6dd2-43cf-b80b-7966c7000762 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:56.775673 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-077bc7bd-6dd2-43cf-b80b-7966c7000762 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:56.776162 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-077bc7bd-6dd2-43cf-b80b-7966c7000762 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:56.776598 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-077bc7bd-6dd2-43cf-b80b-7966c7000762 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:56.925974 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-077bc7bd-6dd2-43cf-b80b-7966c7000762 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:10:56.934258 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-077bc7bd-6dd2-43cf-b80b-7966c7000762 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/80d461e9-eac2-4d78-9e78-d37723f1543b" status: 200 len: 1266 microversion: 2.1 time: 0.171746 Oct 30 01:10:56.934823 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 674/1325] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:10:56 2022] GET /compute/v2.1/servers/80d461e9-eac2-4d78-9e78-d37723f1543b => generated 1266 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:10:57.969506 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-eacda047-c7b8-4cef-81a5-511c85e80fc3 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:10:57.970494 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-eacda047-c7b8-4cef-81a5-511c85e80fc3 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: 80d461e9-eac2-4d78-9e78-d37723f1543b] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:10:57.973353 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell0:SELECT=8 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:57.980425 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-eacda047-c7b8-4cef-81a5-511c85e80fc3 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:57.980898 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-eacda047-c7b8-4cef-81a5-511c85e80fc3 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:57.981326 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-eacda047-c7b8-4cef-81a5-511c85e80fc3 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:57.982008 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-eacda047-c7b8-4cef-81a5-511c85e80fc3 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:57.982443 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-eacda047-c7b8-4cef-81a5-511c85e80fc3 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:57.982852 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-eacda047-c7b8-4cef-81a5-511c85e80fc3 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:58.120317 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-eacda047-c7b8-4cef-81a5-511c85e80fc3 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:10:58.128801 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-eacda047-c7b8-4cef-81a5-511c85e80fc3 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/80d461e9-eac2-4d78-9e78-d37723f1543b" status: 200 len: 1266 microversion: 2.1 time: 0.165832 Oct 30 01:10:58.129376 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 652/1326] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:10:57 2022] GET /compute/v2.1/servers/80d461e9-eac2-4d78-9e78-d37723f1543b => generated 1266 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:10:59.156146 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-2b2bac00-3347-4c9a-9824-64b72eb17b0d tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:10:59.157017 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-2b2bac00-3347-4c9a-9824-64b72eb17b0d tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: 80d461e9-eac2-4d78-9e78-d37723f1543b] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:10:59.160059 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell0:SELECT=4 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:59.167354 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2b2bac00-3347-4c9a-9824-64b72eb17b0d tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:59.167871 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2b2bac00-3347-4c9a-9824-64b72eb17b0d tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:59.168309 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2b2bac00-3347-4c9a-9824-64b72eb17b0d tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:59.169006 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2b2bac00-3347-4c9a-9824-64b72eb17b0d tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:10:59.169460 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2b2bac00-3347-4c9a-9824-64b72eb17b0d tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:10:59.169883 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-2b2bac00-3347-4c9a-9824-64b72eb17b0d tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:10:59.307447 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-2b2bac00-3347-4c9a-9824-64b72eb17b0d tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:10:59.315522 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-2b2bac00-3347-4c9a-9824-64b72eb17b0d tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/80d461e9-eac2-4d78-9e78-d37723f1543b" status: 200 len: 1266 microversion: 2.1 time: 0.166095 Oct 30 01:10:59.316127 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 675/1327] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:10:59 2022] GET /compute/v2.1/servers/80d461e9-eac2-4d78-9e78-d37723f1543b => generated 1266 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:11:00.338609 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-a0d6bc2c-460f-4a9c-ab5f-6095e76c76a6 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:11:00.339226 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-a0d6bc2c-460f-4a9c-ab5f-6095e76c76a6 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: 80d461e9-eac2-4d78-9e78-d37723f1543b] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:11:00.346168 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a0d6bc2c-460f-4a9c-ab5f-6095e76c76a6 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:11:00.346656 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a0d6bc2c-460f-4a9c-ab5f-6095e76c76a6 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:11:00.347087 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a0d6bc2c-460f-4a9c-ab5f-6095e76c76a6 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:11:00.347819 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a0d6bc2c-460f-4a9c-ab5f-6095e76c76a6 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:11:00.348736 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a0d6bc2c-460f-4a9c-ab5f-6095e76c76a6 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:11:00.348736 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a0d6bc2c-460f-4a9c-ab5f-6095e76c76a6 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:11:00.489801 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-a0d6bc2c-460f-4a9c-ab5f-6095e76c76a6 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:11:00.498069 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-a0d6bc2c-460f-4a9c-ab5f-6095e76c76a6 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/80d461e9-eac2-4d78-9e78-d37723f1543b" status: 200 len: 1266 microversion: 2.1 time: 0.163501 Oct 30 01:11:00.498612 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 653/1328] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:11:00 2022] GET /compute/v2.1/servers/80d461e9-eac2-4d78-9e78-d37723f1543b => generated 1266 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:11:01.525454 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-6b08621b-e68f-4aac-a28e-4f2add52f0ae tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:11:01.526469 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-6b08621b-e68f-4aac-a28e-4f2add52f0ae tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: 80d461e9-eac2-4d78-9e78-d37723f1543b] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:11:01.537603 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6b08621b-e68f-4aac-a28e-4f2add52f0ae tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:11:01.538418 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6b08621b-e68f-4aac-a28e-4f2add52f0ae tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:11:01.539154 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6b08621b-e68f-4aac-a28e-4f2add52f0ae tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:11:01.540365 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6b08621b-e68f-4aac-a28e-4f2add52f0ae tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:11:01.541043 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6b08621b-e68f-4aac-a28e-4f2add52f0ae tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:11:01.541390 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6b08621b-e68f-4aac-a28e-4f2add52f0ae tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:11:01.676039 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-6b08621b-e68f-4aac-a28e-4f2add52f0ae tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:11:01.684539 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-6b08621b-e68f-4aac-a28e-4f2add52f0ae tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/80d461e9-eac2-4d78-9e78-d37723f1543b" status: 200 len: 1266 microversion: 2.1 time: 0.165727 Oct 30 01:11:01.685079 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 676/1329] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:11:01 2022] GET /compute/v2.1/servers/80d461e9-eac2-4d78-9e78-d37723f1543b => generated 1266 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:11:02.706918 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-bb70abed-14c6-48e8-ab4e-fdc590d20279 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:11:02.707519 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-bb70abed-14c6-48e8-ab4e-fdc590d20279 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: 80d461e9-eac2-4d78-9e78-d37723f1543b] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:11:02.713795 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bb70abed-14c6-48e8-ab4e-fdc590d20279 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:11:02.714281 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bb70abed-14c6-48e8-ab4e-fdc590d20279 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:11:02.714707 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bb70abed-14c6-48e8-ab4e-fdc590d20279 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:11:02.715386 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bb70abed-14c6-48e8-ab4e-fdc590d20279 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:11:02.715869 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bb70abed-14c6-48e8-ab4e-fdc590d20279 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:11:02.716320 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-bb70abed-14c6-48e8-ab4e-fdc590d20279 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:11:02.851603 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-bb70abed-14c6-48e8-ab4e-fdc590d20279 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:11:02.860318 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-bb70abed-14c6-48e8-ab4e-fdc590d20279 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/80d461e9-eac2-4d78-9e78-d37723f1543b" status: 200 len: 1266 microversion: 2.1 time: 0.157800 Oct 30 01:11:02.860900 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 654/1330] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:11:02 2022] GET /compute/v2.1/servers/80d461e9-eac2-4d78-9e78-d37723f1543b => generated 1266 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:11:03.887309 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-a83645ee-fd95-4bda-9511-31e7aa7c74f3 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:11:03.888300 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-a83645ee-fd95-4bda-9511-31e7aa7c74f3 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: 80d461e9-eac2-4d78-9e78-d37723f1543b] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:11:03.899022 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a83645ee-fd95-4bda-9511-31e7aa7c74f3 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:11:03.899825 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a83645ee-fd95-4bda-9511-31e7aa7c74f3 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:11:03.900555 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a83645ee-fd95-4bda-9511-31e7aa7c74f3 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:11:03.901384 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a83645ee-fd95-4bda-9511-31e7aa7c74f3 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:11:03.901853 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a83645ee-fd95-4bda-9511-31e7aa7c74f3 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:11:03.902269 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a83645ee-fd95-4bda-9511-31e7aa7c74f3 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:11:04.048838 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-a83645ee-fd95-4bda-9511-31e7aa7c74f3 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:11:04.057436 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-a83645ee-fd95-4bda-9511-31e7aa7c74f3 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/80d461e9-eac2-4d78-9e78-d37723f1543b" status: 200 len: 1266 microversion: 2.1 time: 0.176583 Oct 30 01:11:04.058011 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 677/1331] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:11:03 2022] GET /compute/v2.1/servers/80d461e9-eac2-4d78-9e78-d37723f1543b => generated 1266 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:11:04.205627 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [req-57ee4231-e3a2-434c-91ed-7d2871b87ca8 req-ebf8511d-b59d-4516-9b1c-87af087239ba service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "80d461e9-eac2-4d78-9e78-d37723f1543b", "name": "network-vif-plugged", "status": "completed", "tag": "49a23065-cebc-4735-bcae-f52ca512b794"}]} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:11:04.219152 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-57ee4231-e3a2-434c-91ed-7d2871b87ca8 req-ebf8511d-b59d-4516-9b1c-87af087239ba service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:11:04.219624 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-57ee4231-e3a2-434c-91ed-7d2871b87ca8 req-ebf8511d-b59d-4516-9b1c-87af087239ba service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:11:04.220065 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-57ee4231-e3a2-434c-91ed-7d2871b87ca8 req-ebf8511d-b59d-4516-9b1c-87af087239ba service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:11:04.232198 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.compute.server_external_events [req-57ee4231-e3a2-434c-91ed-7d2871b87ca8 req-ebf8511d-b59d-4516-9b1c-87af087239ba service nova] Creating event network-vif-plugged:49a23065-cebc-4735-bcae-f52ca512b794 for instance 80d461e9-eac2-4d78-9e78-d37723f1543b on hv-cinder-maste Oct 30 01:11:04.237713 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [req-57ee4231-e3a2-434c-91ed-7d2871b87ca8 req-ebf8511d-b59d-4516-9b1c-87af087239ba service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035762 Oct 30 01:11:04.238524 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 655/1332] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:11:04 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:11:05.084500 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-7f708aa2-196e-43b7-8d16-30cef73d5ba4 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:11:05.085446 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-7f708aa2-196e-43b7-8d16-30cef73d5ba4 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: 80d461e9-eac2-4d78-9e78-d37723f1543b] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:11:05.096484 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7f708aa2-196e-43b7-8d16-30cef73d5ba4 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:11:05.097302 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7f708aa2-196e-43b7-8d16-30cef73d5ba4 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:11:05.098042 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7f708aa2-196e-43b7-8d16-30cef73d5ba4 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:11:05.099174 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7f708aa2-196e-43b7-8d16-30cef73d5ba4 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:11:05.100068 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7f708aa2-196e-43b7-8d16-30cef73d5ba4 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:11:05.100822 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-7f708aa2-196e-43b7-8d16-30cef73d5ba4 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:11:05.249899 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-7f708aa2-196e-43b7-8d16-30cef73d5ba4 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:11:05.258087 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-7f708aa2-196e-43b7-8d16-30cef73d5ba4 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/80d461e9-eac2-4d78-9e78-d37723f1543b" status: 200 len: 1266 microversion: 2.1 time: 0.180797 Oct 30 01:11:05.258676 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 678/1333] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:11:05 2022] GET /compute/v2.1/servers/80d461e9-eac2-4d78-9e78-d37723f1543b => generated 1266 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:11:06.282700 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-b996fc00-39cd-4b01-a1c9-a01e15fa691e tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:11:06.283541 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-b996fc00-39cd-4b01-a1c9-a01e15fa691e tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: 80d461e9-eac2-4d78-9e78-d37723f1543b] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:11:06.290185 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b996fc00-39cd-4b01-a1c9-a01e15fa691e tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:11:06.290664 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b996fc00-39cd-4b01-a1c9-a01e15fa691e tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:11:06.291093 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b996fc00-39cd-4b01-a1c9-a01e15fa691e tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:11:06.291831 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b996fc00-39cd-4b01-a1c9-a01e15fa691e tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:11:06.292256 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b996fc00-39cd-4b01-a1c9-a01e15fa691e tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:11:06.292667 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-b996fc00-39cd-4b01-a1c9-a01e15fa691e tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:11:06.433387 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-b996fc00-39cd-4b01-a1c9-a01e15fa691e tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:11:06.441609 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-b996fc00-39cd-4b01-a1c9-a01e15fa691e tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/80d461e9-eac2-4d78-9e78-d37723f1543b" status: 200 len: 1454 microversion: 2.1 time: 0.163960 Oct 30 01:11:06.442185 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 656/1334] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:11:06 2022] GET /compute/v2.1/servers/80d461e9-eac2-4d78-9e78-d37723f1543b => generated 1454 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:11:06.464483 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-10ff9cf8-a0e0-4572-a91e-cb387a9675b4 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:11:06.465133 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-10ff9cf8-a0e0-4572-a91e-cb387a9675b4 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: 80d461e9-eac2-4d78-9e78-d37723f1543b] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:11:06.471648 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-10ff9cf8-a0e0-4572-a91e-cb387a9675b4 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:11:06.472316 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-10ff9cf8-a0e0-4572-a91e-cb387a9675b4 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:11:06.472755 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-10ff9cf8-a0e0-4572-a91e-cb387a9675b4 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:11:06.473453 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-10ff9cf8-a0e0-4572-a91e-cb387a9675b4 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:11:06.473900 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-10ff9cf8-a0e0-4572-a91e-cb387a9675b4 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:11:06.474392 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-10ff9cf8-a0e0-4572-a91e-cb387a9675b4 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:11:06.610366 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-10ff9cf8-a0e0-4572-a91e-cb387a9675b4 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:11:06.623230 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-10ff9cf8-a0e0-4572-a91e-cb387a9675b4 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/80d461e9-eac2-4d78-9e78-d37723f1543b" status: 200 len: 1454 microversion: 2.1 time: 0.162391 Oct 30 01:11:06.624164 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 679/1335] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:11:06 2022] GET /compute/v2.1/servers/80d461e9-eac2-4d78-9e78-d37723f1543b => generated 1454 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:11:08.275406 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [req-9e17f0f8-443c-4411-b576-41e881012afd req-ddc22524-6499-4ab3-bbc8-2542816fadd1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "80d461e9-eac2-4d78-9e78-d37723f1543b", "tag": "49a23065-cebc-4735-bcae-f52ca512b794"}]} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:11:08.287109 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-9e17f0f8-443c-4411-b576-41e881012afd req-ddc22524-6499-4ab3-bbc8-2542816fadd1 service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:11:08.287497 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-9e17f0f8-443c-4411-b576-41e881012afd req-ddc22524-6499-4ab3-bbc8-2542816fadd1 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:11:08.288245 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-9e17f0f8-443c-4411-b576-41e881012afd req-ddc22524-6499-4ab3-bbc8-2542816fadd1 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:11:08.301476 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.compute.server_external_events [req-9e17f0f8-443c-4411-b576-41e881012afd req-ddc22524-6499-4ab3-bbc8-2542816fadd1 service nova] Creating event network-changed:49a23065-cebc-4735-bcae-f52ca512b794 for instance 80d461e9-eac2-4d78-9e78-d37723f1543b on hv-cinder-maste Oct 30 01:11:08.306644 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [req-9e17f0f8-443c-4411-b576-41e881012afd req-ddc22524-6499-4ab3-bbc8-2542816fadd1 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036157 Oct 30 01:11:08.307289 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 657/1336] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:11:08 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:12:43.374190 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR oslo.messaging._drivers.impl_rabbit [-] [52b12fb0-913a-4e79-841e-0102bb5650dd] AMQP server on 10.222.0.16:5672 is unreachable: . Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: Oct 30 01:12:43.374765 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_api:SELECT=39,nova_api:UPDATE=1 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:12:43.375572 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell1:SELECT=78,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:12:43.376390 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 30 01:12:43.376577 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Sun Oct 30 01:12:43 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5/action (ip 10.222.0.16) !!! Oct 30 01:12:43.381446 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-5764c2fe-a676-410d-8046-0ec442f102a5 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:12:43.381987 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-5764c2fe-a676-410d-8046-0ec442f102a5 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:12:43.390723 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5764c2fe-a676-410d-8046-0ec442f102a5 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:12:43.391177 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5764c2fe-a676-410d-8046-0ec442f102a5 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:12:43.391613 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5764c2fe-a676-410d-8046-0ec442f102a5 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:12:43.392349 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5764c2fe-a676-410d-8046-0ec442f102a5 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:12:43.392800 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5764c2fe-a676-410d-8046-0ec442f102a5 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:12:43.393224 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-5764c2fe-a676-410d-8046-0ec442f102a5 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:12:43.419118 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.objects.instance [None req-5764c2fe-a676-410d-8046-0ec442f102a5 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lazy-loading 'flavor' on Instance uuid 4444bade-70b8-4cef-9037-02cc5773a5d5 {{(pid=118961) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 30 01:12:43.497389 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-5764c2fe-a676-410d-8046-0ec442f102a5 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "POST /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5/action" status: 202 len: 0 microversion: 2.1 time: 0.125283 Oct 30 01:12:43.498099 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 680/1337] 10.222.0.16 () {60 vars in 1278 bytes} [Sun Oct 30 01:12:43 2022] POST /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5/action => generated 0 bytes in 127 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 30 01:12:43.505162 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_api:SELECT=49,nova_api:INSERT=3 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:12:43.505914 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell1:SELECT=76 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:12:43.506688 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 30 01:12:43.506823 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Sun Oct 30 01:12:43 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5 (ip 10.222.0.16) !!! Oct 30 01:12:43.512977 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-1945f6d5-15ba-44e8-b5aa-97497aae1cdb tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:12:43.513838 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-1945f6d5-15ba-44e8-b5aa-97497aae1cdb tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:12:43.522766 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1945f6d5-15ba-44e8-b5aa-97497aae1cdb tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:12:43.523234 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1945f6d5-15ba-44e8-b5aa-97497aae1cdb tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:12:43.523662 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1945f6d5-15ba-44e8-b5aa-97497aae1cdb tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:12:43.524383 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1945f6d5-15ba-44e8-b5aa-97497aae1cdb tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:12:43.524823 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1945f6d5-15ba-44e8-b5aa-97497aae1cdb tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:12:43.525243 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1945f6d5-15ba-44e8-b5aa-97497aae1cdb tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:12:43.663039 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-1945f6d5-15ba-44e8-b5aa-97497aae1cdb tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baeaad0029f1455497fc3aea467dfc47', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f281bf8af484f7a88df7dc8ac06d887', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:12:43.675231 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-1945f6d5-15ba-44e8-b5aa-97497aae1cdb tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5" status: 200 len: 1570 microversion: 2.1 time: 0.171905 Oct 30 01:12:43.676065 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 658/1338] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:12:43 2022] GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5 => generated 1570 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:12:44.703631 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-1d2fa40d-2071-4fcb-921d-52a35d22d31d tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:12:44.704727 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-1d2fa40d-2071-4fcb-921d-52a35d22d31d tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:12:44.720943 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1d2fa40d-2071-4fcb-921d-52a35d22d31d tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:12:44.721629 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1d2fa40d-2071-4fcb-921d-52a35d22d31d tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:12:44.722172 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1d2fa40d-2071-4fcb-921d-52a35d22d31d tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:12:44.723040 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1d2fa40d-2071-4fcb-921d-52a35d22d31d tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:12:44.723650 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1d2fa40d-2071-4fcb-921d-52a35d22d31d tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:12:44.724197 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-1d2fa40d-2071-4fcb-921d-52a35d22d31d tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:12:44.725048 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO oslo.messaging._drivers.impl_rabbit [-] [52b12fb0-913a-4e79-841e-0102bb5650dd] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 35198. Oct 30 01:12:44.870162 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-1d2fa40d-2071-4fcb-921d-52a35d22d31d tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baeaad0029f1455497fc3aea467dfc47', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f281bf8af484f7a88df7dc8ac06d887', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:12:44.882706 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-1d2fa40d-2071-4fcb-921d-52a35d22d31d tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5" status: 200 len: 1570 microversion: 2.1 time: 0.185697 Oct 30 01:12:44.883464 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 681/1339] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:12:44 2022] GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5 => generated 1570 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:12:45.910356 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-6e6a719e-e1bf-41ea-9fdf-7d98489493d7 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:12:45.911255 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-6e6a719e-e1bf-41ea-9fdf-7d98489493d7 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:12:45.922448 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6e6a719e-e1bf-41ea-9fdf-7d98489493d7 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:12:45.923240 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6e6a719e-e1bf-41ea-9fdf-7d98489493d7 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:12:45.924011 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6e6a719e-e1bf-41ea-9fdf-7d98489493d7 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:12:45.925143 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6e6a719e-e1bf-41ea-9fdf-7d98489493d7 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:12:45.925890 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6e6a719e-e1bf-41ea-9fdf-7d98489493d7 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:12:45.926621 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6e6a719e-e1bf-41ea-9fdf-7d98489493d7 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:12:46.065555 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-6e6a719e-e1bf-41ea-9fdf-7d98489493d7 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baeaad0029f1455497fc3aea467dfc47', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f281bf8af484f7a88df7dc8ac06d887', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:12:46.078307 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-6e6a719e-e1bf-41ea-9fdf-7d98489493d7 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5" status: 200 len: 1570 microversion: 2.1 time: 0.174493 Oct 30 01:12:46.079570 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 659/1340] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:12:45 2022] GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5 => generated 1570 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:12:47.102602 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-b15c1bfd-1847-4b1f-8fb5-ba812267afb8 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:12:47.103227 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-b15c1bfd-1847-4b1f-8fb5-ba812267afb8 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:12:47.110564 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b15c1bfd-1847-4b1f-8fb5-ba812267afb8 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:12:47.111044 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b15c1bfd-1847-4b1f-8fb5-ba812267afb8 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:12:47.111476 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b15c1bfd-1847-4b1f-8fb5-ba812267afb8 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:12:47.112304 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b15c1bfd-1847-4b1f-8fb5-ba812267afb8 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:12:47.112752 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b15c1bfd-1847-4b1f-8fb5-ba812267afb8 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:12:47.113171 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b15c1bfd-1847-4b1f-8fb5-ba812267afb8 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:12:47.257835 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-b15c1bfd-1847-4b1f-8fb5-ba812267afb8 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baeaad0029f1455497fc3aea467dfc47', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f281bf8af484f7a88df7dc8ac06d887', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:12:47.270961 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-b15c1bfd-1847-4b1f-8fb5-ba812267afb8 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5" status: 200 len: 1570 microversion: 2.1 time: 0.172102 Oct 30 01:12:47.271771 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 682/1341] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:12:47 2022] GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5 => generated 1570 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:12:48.301296 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-46493f63-d934-439a-80c9-3273ff949b51 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:12:48.302326 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-46493f63-d934-439a-80c9-3273ff949b51 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:12:48.312368 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-46493f63-d934-439a-80c9-3273ff949b51 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:12:48.312863 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-46493f63-d934-439a-80c9-3273ff949b51 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:12:48.313286 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-46493f63-d934-439a-80c9-3273ff949b51 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:12:48.313991 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-46493f63-d934-439a-80c9-3273ff949b51 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:12:48.314407 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-46493f63-d934-439a-80c9-3273ff949b51 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:12:48.314812 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-46493f63-d934-439a-80c9-3273ff949b51 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:12:48.456277 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-46493f63-d934-439a-80c9-3273ff949b51 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baeaad0029f1455497fc3aea467dfc47', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f281bf8af484f7a88df7dc8ac06d887', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:12:48.469467 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-46493f63-d934-439a-80c9-3273ff949b51 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5" status: 200 len: 1570 microversion: 2.1 time: 0.174343 Oct 30 01:12:48.470214 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 660/1342] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:12:48 2022] GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5 => generated 1570 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:12:49.501662 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-48ae30ef-ceac-4070-aaa8-ba23ae422ac3 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:12:49.502294 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-48ae30ef-ceac-4070-aaa8-ba23ae422ac3 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:12:49.510244 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-48ae30ef-ceac-4070-aaa8-ba23ae422ac3 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:12:49.510808 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-48ae30ef-ceac-4070-aaa8-ba23ae422ac3 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:12:49.511267 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-48ae30ef-ceac-4070-aaa8-ba23ae422ac3 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:12:49.512079 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-48ae30ef-ceac-4070-aaa8-ba23ae422ac3 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:12:49.512586 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-48ae30ef-ceac-4070-aaa8-ba23ae422ac3 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:12:49.513056 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-48ae30ef-ceac-4070-aaa8-ba23ae422ac3 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:12:49.654808 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-48ae30ef-ceac-4070-aaa8-ba23ae422ac3 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baeaad0029f1455497fc3aea467dfc47', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f281bf8af484f7a88df7dc8ac06d887', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:12:49.663277 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-48ae30ef-ceac-4070-aaa8-ba23ae422ac3 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5" status: 200 len: 1570 microversion: 2.1 time: 0.167755 Oct 30 01:12:49.663913 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 683/1343] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:12:49 2022] GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5 => generated 1570 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:12:50.690999 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-6c16e229-3cce-4360-9a70-991e1a299884 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:12:50.691935 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-6c16e229-3cce-4360-9a70-991e1a299884 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:12:50.702708 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6c16e229-3cce-4360-9a70-991e1a299884 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:12:50.703588 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6c16e229-3cce-4360-9a70-991e1a299884 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:12:50.704389 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6c16e229-3cce-4360-9a70-991e1a299884 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:12:50.705120 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6c16e229-3cce-4360-9a70-991e1a299884 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:12:50.705559 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6c16e229-3cce-4360-9a70-991e1a299884 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:12:50.705968 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-6c16e229-3cce-4360-9a70-991e1a299884 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:12:50.854506 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-6c16e229-3cce-4360-9a70-991e1a299884 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baeaad0029f1455497fc3aea467dfc47', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f281bf8af484f7a88df7dc8ac06d887', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:12:50.862613 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-6c16e229-3cce-4360-9a70-991e1a299884 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5" status: 200 len: 1570 microversion: 2.1 time: 0.178190 Oct 30 01:12:50.863181 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 661/1344] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:12:50 2022] GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5 => generated 1570 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:12:51.885585 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-a5ed16f9-6df6-4e03-a93c-21ab39779806 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:12:51.886182 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-a5ed16f9-6df6-4e03-a93c-21ab39779806 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:12:51.893496 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a5ed16f9-6df6-4e03-a93c-21ab39779806 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:12:51.893987 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a5ed16f9-6df6-4e03-a93c-21ab39779806 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:12:51.894428 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a5ed16f9-6df6-4e03-a93c-21ab39779806 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:12:51.895131 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a5ed16f9-6df6-4e03-a93c-21ab39779806 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:12:51.895585 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a5ed16f9-6df6-4e03-a93c-21ab39779806 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:12:51.896045 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a5ed16f9-6df6-4e03-a93c-21ab39779806 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:12:52.035680 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-a5ed16f9-6df6-4e03-a93c-21ab39779806 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baeaad0029f1455497fc3aea467dfc47', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f281bf8af484f7a88df7dc8ac06d887', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:12:52.047952 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-a5ed16f9-6df6-4e03-a93c-21ab39779806 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5" status: 200 len: 1570 microversion: 2.1 time: 0.166152 Oct 30 01:12:52.048842 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 684/1345] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:12:51 2022] GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5 => generated 1570 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:12:53.071891 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-c5f7382f-7559-406b-bfa9-6e97c3b8fbe1 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:12:53.073133 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-c5f7382f-7559-406b-bfa9-6e97c3b8fbe1 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:12:53.081561 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c5f7382f-7559-406b-bfa9-6e97c3b8fbe1 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:12:53.082340 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c5f7382f-7559-406b-bfa9-6e97c3b8fbe1 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:12:53.083011 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c5f7382f-7559-406b-bfa9-6e97c3b8fbe1 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:12:53.084189 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c5f7382f-7559-406b-bfa9-6e97c3b8fbe1 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:12:53.084892 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c5f7382f-7559-406b-bfa9-6e97c3b8fbe1 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:12:53.085600 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c5f7382f-7559-406b-bfa9-6e97c3b8fbe1 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:12:53.241012 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-c5f7382f-7559-406b-bfa9-6e97c3b8fbe1 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baeaad0029f1455497fc3aea467dfc47', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f281bf8af484f7a88df7dc8ac06d887', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:12:53.252496 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-c5f7382f-7559-406b-bfa9-6e97c3b8fbe1 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5" status: 200 len: 1570 microversion: 2.1 time: 0.184576 Oct 30 01:12:53.253079 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 662/1346] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:12:53 2022] GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5 => generated 1570 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:12:54.283191 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-e894ff8a-d2d6-41eb-a26e-f4e8dd91cbcc tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:12:54.283824 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-e894ff8a-d2d6-41eb-a26e-f4e8dd91cbcc tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:12:54.290814 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e894ff8a-d2d6-41eb-a26e-f4e8dd91cbcc tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:12:54.291291 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e894ff8a-d2d6-41eb-a26e-f4e8dd91cbcc tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:12:54.291744 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e894ff8a-d2d6-41eb-a26e-f4e8dd91cbcc tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:12:54.292461 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e894ff8a-d2d6-41eb-a26e-f4e8dd91cbcc tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:12:54.292906 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e894ff8a-d2d6-41eb-a26e-f4e8dd91cbcc tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:12:54.293298 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e894ff8a-d2d6-41eb-a26e-f4e8dd91cbcc tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:12:54.424825 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-e894ff8a-d2d6-41eb-a26e-f4e8dd91cbcc tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baeaad0029f1455497fc3aea467dfc47', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f281bf8af484f7a88df7dc8ac06d887', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:12:54.436213 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-e894ff8a-d2d6-41eb-a26e-f4e8dd91cbcc tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5" status: 200 len: 1570 microversion: 2.1 time: 0.156872 Oct 30 01:12:54.437115 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 685/1347] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:12:54 2022] GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5 => generated 1570 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:12:55.469647 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-a28fc19e-ae64-4c4f-b680-7d72c4789cad tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:12:55.470285 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-a28fc19e-ae64-4c4f-b680-7d72c4789cad tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:12:55.477584 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a28fc19e-ae64-4c4f-b680-7d72c4789cad tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:12:55.478085 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a28fc19e-ae64-4c4f-b680-7d72c4789cad tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:12:55.478539 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a28fc19e-ae64-4c4f-b680-7d72c4789cad tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:12:55.479340 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a28fc19e-ae64-4c4f-b680-7d72c4789cad tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:12:55.479763 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a28fc19e-ae64-4c4f-b680-7d72c4789cad tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:12:55.480211 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a28fc19e-ae64-4c4f-b680-7d72c4789cad tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:12:55.620585 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-a28fc19e-ae64-4c4f-b680-7d72c4789cad tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baeaad0029f1455497fc3aea467dfc47', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f281bf8af484f7a88df7dc8ac06d887', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:12:55.631901 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-a28fc19e-ae64-4c4f-b680-7d72c4789cad tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5" status: 200 len: 1570 microversion: 2.1 time: 0.168939 Oct 30 01:12:55.632752 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 663/1348] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:12:55 2022] GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5 => generated 1570 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:12:56.658172 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-24c82bbf-0c25-4f18-9550-4a0b370aad42 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:12:56.658795 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-24c82bbf-0c25-4f18-9550-4a0b370aad42 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:12:56.665340 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-24c82bbf-0c25-4f18-9550-4a0b370aad42 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:12:56.665827 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-24c82bbf-0c25-4f18-9550-4a0b370aad42 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:12:56.666250 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-24c82bbf-0c25-4f18-9550-4a0b370aad42 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:12:56.666960 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-24c82bbf-0c25-4f18-9550-4a0b370aad42 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:12:56.667412 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-24c82bbf-0c25-4f18-9550-4a0b370aad42 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:12:56.667859 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-24c82bbf-0c25-4f18-9550-4a0b370aad42 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:12:56.800536 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-24c82bbf-0c25-4f18-9550-4a0b370aad42 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baeaad0029f1455497fc3aea467dfc47', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f281bf8af484f7a88df7dc8ac06d887', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:12:56.813422 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-24c82bbf-0c25-4f18-9550-4a0b370aad42 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5" status: 200 len: 1571 microversion: 2.1 time: 0.159045 Oct 30 01:12:56.813994 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 686/1349] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:12:56 2022] GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5 => generated 1571 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:12:56.836331 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-230bdf8f-9063-41b6-b0a5-2aeeab322983 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:12:56.836908 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-230bdf8f-9063-41b6-b0a5-2aeeab322983 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:12:56.843807 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-230bdf8f-9063-41b6-b0a5-2aeeab322983 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:12:56.844506 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-230bdf8f-9063-41b6-b0a5-2aeeab322983 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:12:56.844956 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-230bdf8f-9063-41b6-b0a5-2aeeab322983 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:12:56.845633 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-230bdf8f-9063-41b6-b0a5-2aeeab322983 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:12:56.846094 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-230bdf8f-9063-41b6-b0a5-2aeeab322983 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:12:56.846514 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-230bdf8f-9063-41b6-b0a5-2aeeab322983 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:12:56.883296 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-230bdf8f-9063-41b6-b0a5-2aeeab322983 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=118962) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Oct 30 01:12:56.958098 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-230bdf8f-9063-41b6-b0a5-2aeeab322983 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "POST /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5/action" status: 202 len: 0 microversion: 2.1 time: 0.125939 Oct 30 01:12:56.962069 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 664/1350] 10.222.0.16 () {60 vars in 1278 bytes} [Sun Oct 30 01:12:56 2022] POST /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5/action => generated 0 bytes in 131 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 30 01:12:56.968062 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-0cf7b6b6-4d30-4dfa-8a34-fedb8b2feed5 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:12:56.969225 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-0cf7b6b6-4d30-4dfa-8a34-fedb8b2feed5 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:12:56.976311 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0cf7b6b6-4d30-4dfa-8a34-fedb8b2feed5 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:12:56.976876 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0cf7b6b6-4d30-4dfa-8a34-fedb8b2feed5 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:12:56.977355 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0cf7b6b6-4d30-4dfa-8a34-fedb8b2feed5 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:12:56.978464 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0cf7b6b6-4d30-4dfa-8a34-fedb8b2feed5 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:12:56.978844 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0cf7b6b6-4d30-4dfa-8a34-fedb8b2feed5 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:12:56.979339 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-0cf7b6b6-4d30-4dfa-8a34-fedb8b2feed5 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:12:57.114637 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-0cf7b6b6-4d30-4dfa-8a34-fedb8b2feed5 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baeaad0029f1455497fc3aea467dfc47', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f281bf8af484f7a88df7dc8ac06d887', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:12:57.122766 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-0cf7b6b6-4d30-4dfa-8a34-fedb8b2feed5 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5" status: 200 len: 1519 microversion: 2.1 time: 0.159187 Oct 30 01:12:57.123334 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 687/1351] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:12:56 2022] GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5 => generated 1519 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:12:58.151312 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-7b0b601a-ec5d-46d5-a103-00d1bf381520 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:12:58.152357 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-7b0b601a-ec5d-46d5-a103-00d1bf381520 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:12:58.160505 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7b0b601a-ec5d-46d5-a103-00d1bf381520 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:12:58.160976 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7b0b601a-ec5d-46d5-a103-00d1bf381520 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:12:58.161402 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7b0b601a-ec5d-46d5-a103-00d1bf381520 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:12:58.162133 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7b0b601a-ec5d-46d5-a103-00d1bf381520 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:12:58.162578 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7b0b601a-ec5d-46d5-a103-00d1bf381520 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:12:58.162990 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7b0b601a-ec5d-46d5-a103-00d1bf381520 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:12:58.301230 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-7b0b601a-ec5d-46d5-a103-00d1bf381520 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baeaad0029f1455497fc3aea467dfc47', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f281bf8af484f7a88df7dc8ac06d887', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:12:58.312632 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-7b0b601a-ec5d-46d5-a103-00d1bf381520 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5" status: 200 len: 1521 microversion: 2.1 time: 0.168216 Oct 30 01:12:58.313513 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 665/1352] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:12:58 2022] GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5 => generated 1521 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:12:58.639918 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [req-1e9d6667-492c-4cf6-ad8c-4266a39def35 req-bcf765ed-19cf-4bd5-b412-1f6467685f0b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4444bade-70b8-4cef-9037-02cc5773a5d5", "name": "network-vif-unplugged", "status": "completed", "tag": "4025698b-6028-4fe4-ae38-07266f391e05"}]} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:12:58.651134 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-1e9d6667-492c-4cf6-ad8c-4266a39def35 req-bcf765ed-19cf-4bd5-b412-1f6467685f0b service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:12:58.651980 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-1e9d6667-492c-4cf6-ad8c-4266a39def35 req-bcf765ed-19cf-4bd5-b412-1f6467685f0b service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:12:58.652727 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-1e9d6667-492c-4cf6-ad8c-4266a39def35 req-bcf765ed-19cf-4bd5-b412-1f6467685f0b service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:12:58.666565 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.compute.server_external_events [req-1e9d6667-492c-4cf6-ad8c-4266a39def35 req-bcf765ed-19cf-4bd5-b412-1f6467685f0b service nova] Creating event network-vif-unplugged:4025698b-6028-4fe4-ae38-07266f391e05 for instance 4444bade-70b8-4cef-9037-02cc5773a5d5 on hv-cinder-maste Oct 30 01:12:58.673682 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [req-1e9d6667-492c-4cf6-ad8c-4266a39def35 req-bcf765ed-19cf-4bd5-b412-1f6467685f0b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.037506 Oct 30 01:12:58.674304 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 688/1353] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:12:58 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:12:59.337582 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-629343f3-7d78-47e8-b544-b556a7581b2b tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:12:59.338209 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-629343f3-7d78-47e8-b544-b556a7581b2b tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:12:59.345098 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-629343f3-7d78-47e8-b544-b556a7581b2b tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:12:59.345623 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-629343f3-7d78-47e8-b544-b556a7581b2b tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:12:59.346190 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-629343f3-7d78-47e8-b544-b556a7581b2b tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:12:59.346874 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-629343f3-7d78-47e8-b544-b556a7581b2b tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:12:59.347463 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-629343f3-7d78-47e8-b544-b556a7581b2b tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:12:59.348132 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-629343f3-7d78-47e8-b544-b556a7581b2b tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:12:59.482204 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-629343f3-7d78-47e8-b544-b556a7581b2b tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baeaad0029f1455497fc3aea467dfc47', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f281bf8af484f7a88df7dc8ac06d887', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:12:59.490247 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-629343f3-7d78-47e8-b544-b556a7581b2b tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5" status: 200 len: 1577 microversion: 2.1 time: 0.156643 Oct 30 01:12:59.490850 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 666/1354] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:12:59 2022] GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5 => generated 1577 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:13:00.514069 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-9e3a19c5-6a74-45fe-b608-84bcbdb8807a tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:13:00.514798 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-9e3a19c5-6a74-45fe-b608-84bcbdb8807a tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:13:00.521708 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9e3a19c5-6a74-45fe-b608-84bcbdb8807a tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:00.522218 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9e3a19c5-6a74-45fe-b608-84bcbdb8807a tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:00.522652 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9e3a19c5-6a74-45fe-b608-84bcbdb8807a tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:00.523358 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9e3a19c5-6a74-45fe-b608-84bcbdb8807a tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:00.523859 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9e3a19c5-6a74-45fe-b608-84bcbdb8807a tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:00.524287 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9e3a19c5-6a74-45fe-b608-84bcbdb8807a tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:00.666424 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-9e3a19c5-6a74-45fe-b608-84bcbdb8807a tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baeaad0029f1455497fc3aea467dfc47', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f281bf8af484f7a88df7dc8ac06d887', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:13:00.674968 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-9e3a19c5-6a74-45fe-b608-84bcbdb8807a tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5" status: 200 len: 1577 microversion: 2.1 time: 0.165682 Oct 30 01:13:00.675535 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 689/1355] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:13:00 2022] GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5 => generated 1577 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:13:01.702923 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-c27e8c40-6c07-496f-9a97-56f8474c96c7 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:13:01.704339 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-c27e8c40-6c07-496f-9a97-56f8474c96c7 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:13:01.715283 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c27e8c40-6c07-496f-9a97-56f8474c96c7 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:01.716139 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c27e8c40-6c07-496f-9a97-56f8474c96c7 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:01.716881 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c27e8c40-6c07-496f-9a97-56f8474c96c7 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:01.717953 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c27e8c40-6c07-496f-9a97-56f8474c96c7 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:01.718689 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c27e8c40-6c07-496f-9a97-56f8474c96c7 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:01.719374 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c27e8c40-6c07-496f-9a97-56f8474c96c7 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:01.866118 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-c27e8c40-6c07-496f-9a97-56f8474c96c7 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baeaad0029f1455497fc3aea467dfc47', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f281bf8af484f7a88df7dc8ac06d887', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:13:01.874268 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-c27e8c40-6c07-496f-9a97-56f8474c96c7 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5" status: 200 len: 1577 microversion: 2.1 time: 0.178219 Oct 30 01:13:01.874824 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 667/1356] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:13:01 2022] GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5 => generated 1577 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:13:02.901303 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-b53588d0-aa7b-4c07-b0b7-1c9a4cb0ee86 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:13:02.902227 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-b53588d0-aa7b-4c07-b0b7-1c9a4cb0ee86 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:13:02.913291 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b53588d0-aa7b-4c07-b0b7-1c9a4cb0ee86 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:02.914177 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b53588d0-aa7b-4c07-b0b7-1c9a4cb0ee86 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:02.914886 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b53588d0-aa7b-4c07-b0b7-1c9a4cb0ee86 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:02.916097 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b53588d0-aa7b-4c07-b0b7-1c9a4cb0ee86 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:02.916883 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b53588d0-aa7b-4c07-b0b7-1c9a4cb0ee86 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:02.917660 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b53588d0-aa7b-4c07-b0b7-1c9a4cb0ee86 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:03.063855 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-b53588d0-aa7b-4c07-b0b7-1c9a4cb0ee86 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baeaad0029f1455497fc3aea467dfc47', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f281bf8af484f7a88df7dc8ac06d887', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:13:03.072258 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-b53588d0-aa7b-4c07-b0b7-1c9a4cb0ee86 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5" status: 200 len: 1577 microversion: 2.1 time: 0.177645 Oct 30 01:13:03.072820 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 690/1357] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:13:02 2022] GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5 => generated 1577 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:13:04.100136 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-c01627e4-0b7a-411e-8182-c2ef8d22ad13 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:13:04.101243 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-c01627e4-0b7a-411e-8182-c2ef8d22ad13 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:13:04.111166 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c01627e4-0b7a-411e-8182-c2ef8d22ad13 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:04.111671 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c01627e4-0b7a-411e-8182-c2ef8d22ad13 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:04.112134 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c01627e4-0b7a-411e-8182-c2ef8d22ad13 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:04.112817 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c01627e4-0b7a-411e-8182-c2ef8d22ad13 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:04.113260 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c01627e4-0b7a-411e-8182-c2ef8d22ad13 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:04.113670 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c01627e4-0b7a-411e-8182-c2ef8d22ad13 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:04.257328 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-c01627e4-0b7a-411e-8182-c2ef8d22ad13 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baeaad0029f1455497fc3aea467dfc47', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f281bf8af484f7a88df7dc8ac06d887', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:13:04.265243 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-c01627e4-0b7a-411e-8182-c2ef8d22ad13 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5" status: 200 len: 1577 microversion: 2.1 time: 0.171868 Oct 30 01:13:04.265789 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 668/1358] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:13:04 2022] GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5 => generated 1577 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:13:05.288897 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-9d971a15-080c-4d30-a4ff-4591d2c64cae tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:13:05.289531 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-9d971a15-080c-4d30-a4ff-4591d2c64cae tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:13:05.296586 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9d971a15-080c-4d30-a4ff-4591d2c64cae tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:05.297060 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9d971a15-080c-4d30-a4ff-4591d2c64cae tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:05.297516 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9d971a15-080c-4d30-a4ff-4591d2c64cae tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:05.298267 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9d971a15-080c-4d30-a4ff-4591d2c64cae tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:05.298718 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9d971a15-080c-4d30-a4ff-4591d2c64cae tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:05.299127 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-9d971a15-080c-4d30-a4ff-4591d2c64cae tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:05.442210 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-9d971a15-080c-4d30-a4ff-4591d2c64cae tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baeaad0029f1455497fc3aea467dfc47', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f281bf8af484f7a88df7dc8ac06d887', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:13:05.450410 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-9d971a15-080c-4d30-a4ff-4591d2c64cae tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5" status: 200 len: 1577 microversion: 2.1 time: 0.165444 Oct 30 01:13:05.450991 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 691/1359] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:13:05 2022] GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5 => generated 1577 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:13:06.478064 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-117c65b5-7a2a-486d-a9a4-4c04156be3ab tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:13:06.478985 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-117c65b5-7a2a-486d-a9a4-4c04156be3ab tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:13:06.488600 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-117c65b5-7a2a-486d-a9a4-4c04156be3ab tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:06.489074 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-117c65b5-7a2a-486d-a9a4-4c04156be3ab tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:06.489506 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-117c65b5-7a2a-486d-a9a4-4c04156be3ab tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:06.490192 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-117c65b5-7a2a-486d-a9a4-4c04156be3ab tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:06.490629 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-117c65b5-7a2a-486d-a9a4-4c04156be3ab tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:06.491042 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-117c65b5-7a2a-486d-a9a4-4c04156be3ab tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:06.630220 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-117c65b5-7a2a-486d-a9a4-4c04156be3ab tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baeaad0029f1455497fc3aea467dfc47', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f281bf8af484f7a88df7dc8ac06d887', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:13:06.638343 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-117c65b5-7a2a-486d-a9a4-4c04156be3ab tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5" status: 200 len: 1577 microversion: 2.1 time: 0.166686 Oct 30 01:13:06.638888 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 669/1360] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:13:06 2022] GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5 => generated 1577 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:13:07.663882 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-901546d1-7baf-4908-ab3e-a27392f31651 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:13:07.664495 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-901546d1-7baf-4908-ab3e-a27392f31651 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:13:07.671037 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-901546d1-7baf-4908-ab3e-a27392f31651 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:07.671565 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-901546d1-7baf-4908-ab3e-a27392f31651 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:07.672010 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-901546d1-7baf-4908-ab3e-a27392f31651 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:07.672704 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-901546d1-7baf-4908-ab3e-a27392f31651 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:07.673157 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-901546d1-7baf-4908-ab3e-a27392f31651 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:07.673579 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-901546d1-7baf-4908-ab3e-a27392f31651 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:07.811426 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-901546d1-7baf-4908-ab3e-a27392f31651 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baeaad0029f1455497fc3aea467dfc47', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f281bf8af484f7a88df7dc8ac06d887', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:13:07.820029 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-901546d1-7baf-4908-ab3e-a27392f31651 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5" status: 200 len: 1577 microversion: 2.1 time: 0.160052 Oct 30 01:13:07.820613 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 692/1361] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:13:07 2022] GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5 => generated 1577 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:13:08.845710 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-130dcd71-02a4-4cef-aed8-4168219396c0 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:13:08.846327 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-130dcd71-02a4-4cef-aed8-4168219396c0 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:13:08.853207 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-130dcd71-02a4-4cef-aed8-4168219396c0 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:08.853708 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-130dcd71-02a4-4cef-aed8-4168219396c0 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:08.854134 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-130dcd71-02a4-4cef-aed8-4168219396c0 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:08.854819 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-130dcd71-02a4-4cef-aed8-4168219396c0 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:08.855262 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-130dcd71-02a4-4cef-aed8-4168219396c0 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:08.855673 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-130dcd71-02a4-4cef-aed8-4168219396c0 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:08.993565 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-130dcd71-02a4-4cef-aed8-4168219396c0 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baeaad0029f1455497fc3aea467dfc47', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f281bf8af484f7a88df7dc8ac06d887', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:13:09.001758 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-130dcd71-02a4-4cef-aed8-4168219396c0 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5" status: 200 len: 1577 microversion: 2.1 time: 0.160044 Oct 30 01:13:09.002462 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 670/1362] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:13:08 2022] GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5 => generated 1577 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:13:10.029893 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-68485a07-01c7-4e8e-9795-18cfdf214886 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:13:10.030902 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-68485a07-01c7-4e8e-9795-18cfdf214886 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:13:10.039048 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-68485a07-01c7-4e8e-9795-18cfdf214886 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:10.039530 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-68485a07-01c7-4e8e-9795-18cfdf214886 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:10.039990 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-68485a07-01c7-4e8e-9795-18cfdf214886 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:10.040681 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-68485a07-01c7-4e8e-9795-18cfdf214886 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:10.041126 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-68485a07-01c7-4e8e-9795-18cfdf214886 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:10.041545 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-68485a07-01c7-4e8e-9795-18cfdf214886 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:10.189153 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-68485a07-01c7-4e8e-9795-18cfdf214886 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baeaad0029f1455497fc3aea467dfc47', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f281bf8af484f7a88df7dc8ac06d887', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:13:10.197264 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-68485a07-01c7-4e8e-9795-18cfdf214886 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5" status: 200 len: 1577 microversion: 2.1 time: 0.173948 Oct 30 01:13:10.197812 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 693/1363] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:13:10 2022] GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5 => generated 1577 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:13:11.224119 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-12f157fd-8660-45f4-8bc6-147fa7dfc2d8 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:13:11.224702 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-12f157fd-8660-45f4-8bc6-147fa7dfc2d8 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:13:11.231666 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-12f157fd-8660-45f4-8bc6-147fa7dfc2d8 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:11.232229 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-12f157fd-8660-45f4-8bc6-147fa7dfc2d8 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:11.232665 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-12f157fd-8660-45f4-8bc6-147fa7dfc2d8 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:11.233483 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-12f157fd-8660-45f4-8bc6-147fa7dfc2d8 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:11.233980 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-12f157fd-8660-45f4-8bc6-147fa7dfc2d8 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:11.234440 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-12f157fd-8660-45f4-8bc6-147fa7dfc2d8 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:11.366513 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-12f157fd-8660-45f4-8bc6-147fa7dfc2d8 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baeaad0029f1455497fc3aea467dfc47', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f281bf8af484f7a88df7dc8ac06d887', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:13:11.374552 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-12f157fd-8660-45f4-8bc6-147fa7dfc2d8 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5" status: 200 len: 1577 microversion: 2.1 time: 0.155865 Oct 30 01:13:11.375113 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 671/1364] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:13:11 2022] GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5 => generated 1577 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:13:12.397122 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-21af727f-af65-46c8-b947-a6f0f4d14b3f tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:13:12.397760 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-21af727f-af65-46c8-b947-a6f0f4d14b3f tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:13:12.404329 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-21af727f-af65-46c8-b947-a6f0f4d14b3f tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:12.404813 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-21af727f-af65-46c8-b947-a6f0f4d14b3f tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:12.405249 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-21af727f-af65-46c8-b947-a6f0f4d14b3f tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:12.405944 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-21af727f-af65-46c8-b947-a6f0f4d14b3f tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:12.406413 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-21af727f-af65-46c8-b947-a6f0f4d14b3f tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:12.406847 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-21af727f-af65-46c8-b947-a6f0f4d14b3f tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:12.547792 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-21af727f-af65-46c8-b947-a6f0f4d14b3f tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baeaad0029f1455497fc3aea467dfc47', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f281bf8af484f7a88df7dc8ac06d887', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:13:12.556152 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-21af727f-af65-46c8-b947-a6f0f4d14b3f tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5" status: 200 len: 1577 microversion: 2.1 time: 0.163037 Oct 30 01:13:12.556800 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 694/1365] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:13:12 2022] GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5 => generated 1577 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 01:13:13.583340 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-c1b6f71b-15ee-4ef0-9e56-7418b320aeb0 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:13:13.584142 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-c1b6f71b-15ee-4ef0-9e56-7418b320aeb0 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:13:13.587312 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_api:SELECT=33,nova_api:UPDATE=1 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:13:13.594980 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c1b6f71b-15ee-4ef0-9e56-7418b320aeb0 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:13.595650 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c1b6f71b-15ee-4ef0-9e56-7418b320aeb0 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:13.596287 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c1b6f71b-15ee-4ef0-9e56-7418b320aeb0 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:13.597186 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c1b6f71b-15ee-4ef0-9e56-7418b320aeb0 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:13.597629 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c1b6f71b-15ee-4ef0-9e56-7418b320aeb0 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:13.598036 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c1b6f71b-15ee-4ef0-9e56-7418b320aeb0 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:13.601554 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell1:SELECT=89,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:13:13.732826 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-c1b6f71b-15ee-4ef0-9e56-7418b320aeb0 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baeaad0029f1455497fc3aea467dfc47', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f281bf8af484f7a88df7dc8ac06d887', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:13:13.744910 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-c1b6f71b-15ee-4ef0-9e56-7418b320aeb0 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5" status: 200 len: 1577 microversion: 2.1 time: 0.167815 Oct 30 01:13:13.745798 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 672/1366] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:13:13 2022] GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5 => generated 1577 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:13:14.012332 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [req-b12d0082-141b-425f-86d5-1bb73924903f req-004cc9ad-5eea-4e28-aa7d-cf7c250f35e4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4444bade-70b8-4cef-9037-02cc5773a5d5", "name": "network-vif-plugged", "status": "completed", "tag": "4025698b-6028-4fe4-ae38-07266f391e05"}]} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:13:14.017366 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_api:SELECT=31 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:13:14.023493 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-b12d0082-141b-425f-86d5-1bb73924903f req-004cc9ad-5eea-4e28-aa7d-cf7c250f35e4 service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:14.024017 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-b12d0082-141b-425f-86d5-1bb73924903f req-004cc9ad-5eea-4e28-aa7d-cf7c250f35e4 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:14.024464 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-b12d0082-141b-425f-86d5-1bb73924903f req-004cc9ad-5eea-4e28-aa7d-cf7c250f35e4 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:14.027092 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:13:14.037255 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.compute.server_external_events [req-b12d0082-141b-425f-86d5-1bb73924903f req-004cc9ad-5eea-4e28-aa7d-cf7c250f35e4 service nova] Creating event network-vif-plugged:4025698b-6028-4fe4-ae38-07266f391e05 for instance 4444bade-70b8-4cef-9037-02cc5773a5d5 on hv-cinder-maste Oct 30 01:13:14.042762 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [req-b12d0082-141b-425f-86d5-1bb73924903f req-004cc9ad-5eea-4e28-aa7d-cf7c250f35e4 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.091499 Oct 30 01:13:14.043364 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 695/1367] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:13:13 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:13:14.775144 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-d59019e8-643c-4257-9c4b-6155c0375df5 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:13:14.776135 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-d59019e8-643c-4257-9c4b-6155c0375df5 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:13:14.787360 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d59019e8-643c-4257-9c4b-6155c0375df5 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:14.788196 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d59019e8-643c-4257-9c4b-6155c0375df5 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:14.788902 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d59019e8-643c-4257-9c4b-6155c0375df5 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:14.790032 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d59019e8-643c-4257-9c4b-6155c0375df5 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:14.790782 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d59019e8-643c-4257-9c4b-6155c0375df5 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:14.791196 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d59019e8-643c-4257-9c4b-6155c0375df5 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:14.926431 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-d59019e8-643c-4257-9c4b-6155c0375df5 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baeaad0029f1455497fc3aea467dfc47', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f281bf8af484f7a88df7dc8ac06d887', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:13:14.934634 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-d59019e8-643c-4257-9c4b-6155c0375df5 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5" status: 200 len: 1577 microversion: 2.1 time: 0.166138 Oct 30 01:13:14.935205 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 673/1368] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:13:14 2022] GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5 => generated 1577 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:13:15.960657 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-06252fb4-e1fc-4d4a-98a6-f3912710c7c3 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:13:15.961471 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-06252fb4-e1fc-4d4a-98a6-f3912710c7c3 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:13:15.968080 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-06252fb4-e1fc-4d4a-98a6-f3912710c7c3 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:15.968569 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-06252fb4-e1fc-4d4a-98a6-f3912710c7c3 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:15.969005 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-06252fb4-e1fc-4d4a-98a6-f3912710c7c3 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:15.969707 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-06252fb4-e1fc-4d4a-98a6-f3912710c7c3 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:15.970154 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-06252fb4-e1fc-4d4a-98a6-f3912710c7c3 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:15.970573 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-06252fb4-e1fc-4d4a-98a6-f3912710c7c3 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:16.103017 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-06252fb4-e1fc-4d4a-98a6-f3912710c7c3 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baeaad0029f1455497fc3aea467dfc47', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f281bf8af484f7a88df7dc8ac06d887', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:13:16.111119 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-06252fb4-e1fc-4d4a-98a6-f3912710c7c3 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5" status: 200 len: 1577 microversion: 2.1 time: 0.156572 Oct 30 01:13:16.111752 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 696/1369] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:13:15 2022] GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5 => generated 1577 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:13:17.135369 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-d98c8b0a-5ed2-4842-bf4c-57aa1cd607a0 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:13:17.136130 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-d98c8b0a-5ed2-4842-bf4c-57aa1cd607a0 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:13:17.143134 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d98c8b0a-5ed2-4842-bf4c-57aa1cd607a0 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:17.143653 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d98c8b0a-5ed2-4842-bf4c-57aa1cd607a0 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:17.144117 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d98c8b0a-5ed2-4842-bf4c-57aa1cd607a0 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:17.144821 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d98c8b0a-5ed2-4842-bf4c-57aa1cd607a0 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:17.145302 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d98c8b0a-5ed2-4842-bf4c-57aa1cd607a0 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:17.145721 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d98c8b0a-5ed2-4842-bf4c-57aa1cd607a0 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:17.285480 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-d98c8b0a-5ed2-4842-bf4c-57aa1cd607a0 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baeaad0029f1455497fc3aea467dfc47', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f281bf8af484f7a88df7dc8ac06d887', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:13:17.293549 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-d98c8b0a-5ed2-4842-bf4c-57aa1cd607a0 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5" status: 200 len: 1564 microversion: 2.1 time: 0.162377 Oct 30 01:13:17.294091 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 674/1370] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:13:17 2022] GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5 => generated 1564 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:13:19.972708 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-a6092a9a-d5d1-4ba1-a728-034b8d73537c tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-944175144", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1884997751", "security_groups": [{"name": "tempest-secgroup-smoke-1108501088"}], "block_device_mapping_v2": [{"uuid": "c70ab05d-861d-4f56-859c-3ac0499bf65a", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "c1a88451-f287-4e35-b7e8-a6d4e4aaa603"}]}} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:13:20.095567 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.network.neutron [None req-a6092a9a-d5d1-4ba1-a728-034b8d73537c tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] validate_networks() for [('c1a88451-f287-4e35-b7e8-a6d4e4aaa603', None, None, None, None, None)] {{(pid=118961) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 30 01:13:20.757259 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-a6092a9a-d5d1-4ba1-a728-034b8d73537c tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Flavor limits 0:0:0 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 30 01:13:20.757686 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-a6092a9a-d5d1-4ba1-a728-034b8d73537c tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Image limits 0:0:0 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 30 01:13:20.758160 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-a6092a9a-d5d1-4ba1-a728-034b8d73537c tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Flavor pref 0:0:0 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 30 01:13:20.758568 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-a6092a9a-d5d1-4ba1-a728-034b8d73537c tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Image pref 0:0:0 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 30 01:13:20.758972 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.virt.hardware [None req-a6092a9a-d5d1-4ba1-a728-034b8d73537c tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118961) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 30 01:13:20.768899 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a6092a9a-d5d1-4ba1-a728-034b8d73537c tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:20.769387 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a6092a9a-d5d1-4ba1-a728-034b8d73537c tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:20.769816 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a6092a9a-d5d1-4ba1-a728-034b8d73537c tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:20.770544 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a6092a9a-d5d1-4ba1-a728-034b8d73537c tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:20.770997 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a6092a9a-d5d1-4ba1-a728-034b8d73537c tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:20.771421 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-a6092a9a-d5d1-4ba1-a728-034b8d73537c tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:20.789056 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.quota [None req-a6092a9a-d5d1-4ba1-a728-034b8d73537c tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Getting quotas for project 238d71be4e094cb6b84f9a48a8a2c79c. Resources: {'ram', 'instances', 'cores'} {{(pid=118961) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 30 01:13:20.793709 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.quota [None req-a6092a9a-d5d1-4ba1-a728-034b8d73537c tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Getting quotas for user 96e0cd00e4ef4a2da960aa5d4f66972a and project 238d71be4e094cb6b84f9a48a8a2c79c. Resources: {'ram', 'instances', 'cores'} {{(pid=118961) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 30 01:13:20.852797 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-a6092a9a-d5d1-4ba1-a728-034b8d73537c tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Going to run 1 instances... {{(pid=118961) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 30 01:13:20.854930 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-a6092a9a-d5d1-4ba1-a728-034b8d73537c tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: c7507aca-90a7-46ed-9bf0-2a162db3bfc7] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='c70ab05d-861d-4f56-859c-3ac0499bf65a',volume_size=None,volume_type=None)] {{(pid=118961) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 30 01:13:20.992871 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR oslo.messaging._drivers.impl_rabbit [None req-a6092a9a-d5d1-4ba1-a728-034b8d73537c tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [b36edcff-6649-43d9-a913-81fa9ddad5ca] AMQP server on 10.222.0.16:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Oct 30 01:13:22.005750 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO oslo.messaging._drivers.impl_rabbit [None req-a6092a9a-d5d1-4ba1-a728-034b8d73537c tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [b36edcff-6649-43d9-a913-81fa9ddad5ca] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 35342. Oct 30 01:13:22.011705 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-a6092a9a-d5d1-4ba1-a728-034b8d73537c tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 2.045650 Oct 30 01:13:22.015807 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 697/1371] 10.222.0.16 () {60 vars in 1147 bytes} [Sun Oct 30 01:13:19 2022] POST /compute/v2.1/servers => generated 406 bytes in 2047 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 30 01:13:22.024392 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-2c65fd14-122d-4fed-ae1f-0b54c5dea3a0 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:13:22.024996 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-2c65fd14-122d-4fed-ae1f-0b54c5dea3a0 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: c7507aca-90a7-46ed-9bf0-2a162db3bfc7] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:13:22.078975 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-2c65fd14-122d-4fed-ae1f-0b54c5dea3a0 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:13:22.086001 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-2c65fd14-122d-4fed-ae1f-0b54c5dea3a0 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/c7507aca-90a7-46ed-9bf0-2a162db3bfc7" status: 200 len: 1094 microversion: 2.1 time: 0.066186 Oct 30 01:13:22.086561 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 675/1372] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:13:22 2022] GET /compute/v2.1/servers/c7507aca-90a7-46ed-9bf0-2a162db3bfc7 => generated 1094 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:13:23.109299 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-26773f68-bfb2-47b6-800d-92f885236a1d tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:13:23.109877 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-26773f68-bfb2-47b6-800d-92f885236a1d tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: c7507aca-90a7-46ed-9bf0-2a162db3bfc7] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:13:23.116286 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-26773f68-bfb2-47b6-800d-92f885236a1d tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:23.116770 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-26773f68-bfb2-47b6-800d-92f885236a1d tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:23.117207 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-26773f68-bfb2-47b6-800d-92f885236a1d tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:23.117900 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-26773f68-bfb2-47b6-800d-92f885236a1d tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:23.118349 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-26773f68-bfb2-47b6-800d-92f885236a1d tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:23.118977 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-26773f68-bfb2-47b6-800d-92f885236a1d tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:23.186789 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-26773f68-bfb2-47b6-800d-92f885236a1d tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:13:23.195243 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-26773f68-bfb2-47b6-800d-92f885236a1d tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/c7507aca-90a7-46ed-9bf0-2a162db3bfc7" status: 200 len: 1192 microversion: 2.1 time: 0.090504 Oct 30 01:13:23.195853 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 698/1373] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:13:23 2022] GET /compute/v2.1/servers/c7507aca-90a7-46ed-9bf0-2a162db3bfc7 => generated 1192 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:13:24.218653 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-46efdd5c-aea8-46ba-93de-48cbc6f7034f tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:13:24.219449 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-46efdd5c-aea8-46ba-93de-48cbc6f7034f tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: c7507aca-90a7-46ed-9bf0-2a162db3bfc7] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:13:24.227140 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-46efdd5c-aea8-46ba-93de-48cbc6f7034f tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:24.227630 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-46efdd5c-aea8-46ba-93de-48cbc6f7034f tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:24.228155 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-46efdd5c-aea8-46ba-93de-48cbc6f7034f tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:24.228856 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-46efdd5c-aea8-46ba-93de-48cbc6f7034f tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:24.229305 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-46efdd5c-aea8-46ba-93de-48cbc6f7034f tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:24.229725 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-46efdd5c-aea8-46ba-93de-48cbc6f7034f tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:24.377063 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-46efdd5c-aea8-46ba-93de-48cbc6f7034f tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:13:24.385120 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-46efdd5c-aea8-46ba-93de-48cbc6f7034f tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/c7507aca-90a7-46ed-9bf0-2a162db3bfc7" status: 200 len: 1266 microversion: 2.1 time: 0.170331 Oct 30 01:13:24.385815 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 676/1374] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:13:24 2022] GET /compute/v2.1/servers/c7507aca-90a7-46ed-9bf0-2a162db3bfc7 => generated 1266 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:13:24.972624 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [req-b159c0e7-87b7-421a-ac6e-47e068c963ce req-e63e77a9-e5eb-4259-8e12-7be343e4f902 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c7507aca-90a7-46ed-9bf0-2a162db3bfc7", "tag": "a40ffbd8-6a20-454e-a5cf-5c6347862ffe"}]} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:13:24.982836 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-b159c0e7-87b7-421a-ac6e-47e068c963ce req-e63e77a9-e5eb-4259-8e12-7be343e4f902 service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:24.983310 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-b159c0e7-87b7-421a-ac6e-47e068c963ce req-e63e77a9-e5eb-4259-8e12-7be343e4f902 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:24.983741 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-b159c0e7-87b7-421a-ac6e-47e068c963ce req-e63e77a9-e5eb-4259-8e12-7be343e4f902 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:24.997486 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.compute.server_external_events [req-b159c0e7-87b7-421a-ac6e-47e068c963ce req-e63e77a9-e5eb-4259-8e12-7be343e4f902 service nova] Creating event network-changed:a40ffbd8-6a20-454e-a5cf-5c6347862ffe for instance c7507aca-90a7-46ed-9bf0-2a162db3bfc7 on hv-cinder-maste Oct 30 01:13:25.002505 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [req-b159c0e7-87b7-421a-ac6e-47e068c963ce req-e63e77a9-e5eb-4259-8e12-7be343e4f902 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.093792 Oct 30 01:13:25.003350 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 699/1375] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:13:24 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:13:25.409478 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-e7c291c0-6a80-4a5b-912a-9278d7d598f8 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:13:25.410090 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-e7c291c0-6a80-4a5b-912a-9278d7d598f8 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: c7507aca-90a7-46ed-9bf0-2a162db3bfc7] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:13:25.420099 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e7c291c0-6a80-4a5b-912a-9278d7d598f8 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:25.420822 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e7c291c0-6a80-4a5b-912a-9278d7d598f8 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:25.421476 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e7c291c0-6a80-4a5b-912a-9278d7d598f8 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:25.422545 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e7c291c0-6a80-4a5b-912a-9278d7d598f8 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:25.423239 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e7c291c0-6a80-4a5b-912a-9278d7d598f8 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:25.423893 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e7c291c0-6a80-4a5b-912a-9278d7d598f8 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:25.571741 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-e7c291c0-6a80-4a5b-912a-9278d7d598f8 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:13:25.583943 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-e7c291c0-6a80-4a5b-912a-9278d7d598f8 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/c7507aca-90a7-46ed-9bf0-2a162db3bfc7" status: 200 len: 1266 microversion: 2.1 time: 0.178710 Oct 30 01:13:25.584782 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 677/1376] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:13:25 2022] GET /compute/v2.1/servers/c7507aca-90a7-46ed-9bf0-2a162db3bfc7 => generated 1266 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:13:26.616425 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-d2f190aa-dc61-47a3-839c-c404e560905f tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:13:26.617020 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-d2f190aa-dc61-47a3-839c-c404e560905f tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: c7507aca-90a7-46ed-9bf0-2a162db3bfc7] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:13:26.625326 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d2f190aa-dc61-47a3-839c-c404e560905f tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:26.625968 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d2f190aa-dc61-47a3-839c-c404e560905f tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:26.626548 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d2f190aa-dc61-47a3-839c-c404e560905f tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:26.627596 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d2f190aa-dc61-47a3-839c-c404e560905f tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:26.628097 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d2f190aa-dc61-47a3-839c-c404e560905f tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:26.628628 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-d2f190aa-dc61-47a3-839c-c404e560905f tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:26.760760 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-d2f190aa-dc61-47a3-839c-c404e560905f tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:13:26.768734 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-d2f190aa-dc61-47a3-839c-c404e560905f tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/c7507aca-90a7-46ed-9bf0-2a162db3bfc7" status: 200 len: 1266 microversion: 2.1 time: 0.157294 Oct 30 01:13:26.769309 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 700/1377] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:13:26 2022] GET /compute/v2.1/servers/c7507aca-90a7-46ed-9bf0-2a162db3bfc7 => generated 1266 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:13:27.796471 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-a8e6255e-21cc-4364-a1f4-42a16921f762 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:13:27.797418 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-a8e6255e-21cc-4364-a1f4-42a16921f762 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: c7507aca-90a7-46ed-9bf0-2a162db3bfc7] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:13:27.807570 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a8e6255e-21cc-4364-a1f4-42a16921f762 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:27.808089 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a8e6255e-21cc-4364-a1f4-42a16921f762 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:27.808523 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a8e6255e-21cc-4364-a1f4-42a16921f762 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:27.809210 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a8e6255e-21cc-4364-a1f4-42a16921f762 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:27.809654 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a8e6255e-21cc-4364-a1f4-42a16921f762 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:27.810080 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a8e6255e-21cc-4364-a1f4-42a16921f762 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:27.958366 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-a8e6255e-21cc-4364-a1f4-42a16921f762 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:13:27.970476 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-a8e6255e-21cc-4364-a1f4-42a16921f762 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/c7507aca-90a7-46ed-9bf0-2a162db3bfc7" status: 200 len: 1266 microversion: 2.1 time: 0.180196 Oct 30 01:13:27.971265 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 678/1378] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:13:27 2022] GET /compute/v2.1/servers/c7507aca-90a7-46ed-9bf0-2a162db3bfc7 => generated 1266 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:13:29.000099 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-471e268e-606b-443c-aa01-83cf3f8f7ea0 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:13:29.001192 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-471e268e-606b-443c-aa01-83cf3f8f7ea0 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: c7507aca-90a7-46ed-9bf0-2a162db3bfc7] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:13:29.011107 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-471e268e-606b-443c-aa01-83cf3f8f7ea0 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:29.011629 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-471e268e-606b-443c-aa01-83cf3f8f7ea0 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:29.012100 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-471e268e-606b-443c-aa01-83cf3f8f7ea0 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:29.012822 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-471e268e-606b-443c-aa01-83cf3f8f7ea0 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:29.013271 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-471e268e-606b-443c-aa01-83cf3f8f7ea0 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:29.013698 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-471e268e-606b-443c-aa01-83cf3f8f7ea0 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:29.148107 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-471e268e-606b-443c-aa01-83cf3f8f7ea0 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:13:29.157134 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-471e268e-606b-443c-aa01-83cf3f8f7ea0 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/c7507aca-90a7-46ed-9bf0-2a162db3bfc7" status: 200 len: 1266 microversion: 2.1 time: 0.163781 Oct 30 01:13:29.157707 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 701/1379] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:13:28 2022] GET /compute/v2.1/servers/c7507aca-90a7-46ed-9bf0-2a162db3bfc7 => generated 1266 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:13:30.182721 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-8dc87206-b8ab-4ecb-8772-d648439a33ed tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:13:30.183347 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-8dc87206-b8ab-4ecb-8772-d648439a33ed tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: c7507aca-90a7-46ed-9bf0-2a162db3bfc7] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:13:30.190202 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8dc87206-b8ab-4ecb-8772-d648439a33ed tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:30.190690 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8dc87206-b8ab-4ecb-8772-d648439a33ed tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:30.191125 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8dc87206-b8ab-4ecb-8772-d648439a33ed tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:30.191871 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8dc87206-b8ab-4ecb-8772-d648439a33ed tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:30.192323 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8dc87206-b8ab-4ecb-8772-d648439a33ed tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:30.192748 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-8dc87206-b8ab-4ecb-8772-d648439a33ed tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:30.327135 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-8dc87206-b8ab-4ecb-8772-d648439a33ed tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:13:30.335409 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-8dc87206-b8ab-4ecb-8772-d648439a33ed tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/c7507aca-90a7-46ed-9bf0-2a162db3bfc7" status: 200 len: 1266 microversion: 2.1 time: 0.157874 Oct 30 01:13:30.336019 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 679/1380] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:13:30 2022] GET /compute/v2.1/servers/c7507aca-90a7-46ed-9bf0-2a162db3bfc7 => generated 1266 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:13:31.362246 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-ba3cccaa-e59c-4f80-a84d-9ac403d8ecd3 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:13:31.363271 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-ba3cccaa-e59c-4f80-a84d-9ac403d8ecd3 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: c7507aca-90a7-46ed-9bf0-2a162db3bfc7] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:13:31.367738 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell0:SELECT=2 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:13:31.368647 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell0:SELECT=6 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:13:31.378358 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ba3cccaa-e59c-4f80-a84d-9ac403d8ecd3 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:31.378862 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ba3cccaa-e59c-4f80-a84d-9ac403d8ecd3 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:31.379445 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ba3cccaa-e59c-4f80-a84d-9ac403d8ecd3 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:31.380279 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ba3cccaa-e59c-4f80-a84d-9ac403d8ecd3 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:31.380898 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ba3cccaa-e59c-4f80-a84d-9ac403d8ecd3 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:31.381476 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-ba3cccaa-e59c-4f80-a84d-9ac403d8ecd3 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:31.523428 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-ba3cccaa-e59c-4f80-a84d-9ac403d8ecd3 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:13:31.532085 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-ba3cccaa-e59c-4f80-a84d-9ac403d8ecd3 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/c7507aca-90a7-46ed-9bf0-2a162db3bfc7" status: 200 len: 1266 microversion: 2.1 time: 0.176298 Oct 30 01:13:31.532640 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 702/1381] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:13:31 2022] GET /compute/v2.1/servers/c7507aca-90a7-46ed-9bf0-2a162db3bfc7 => generated 1266 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:13:32.558870 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-60145ccc-2c0d-486d-8c8c-7ac3558cfd33 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:13:32.559828 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-60145ccc-2c0d-486d-8c8c-7ac3558cfd33 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: c7507aca-90a7-46ed-9bf0-2a162db3bfc7] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:13:32.562548 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell0:SELECT=2 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:13:32.570766 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-60145ccc-2c0d-486d-8c8c-7ac3558cfd33 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:32.571491 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-60145ccc-2c0d-486d-8c8c-7ac3558cfd33 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:32.572185 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-60145ccc-2c0d-486d-8c8c-7ac3558cfd33 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:32.573220 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-60145ccc-2c0d-486d-8c8c-7ac3558cfd33 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:32.573658 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-60145ccc-2c0d-486d-8c8c-7ac3558cfd33 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:32.574069 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-60145ccc-2c0d-486d-8c8c-7ac3558cfd33 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:32.712298 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-60145ccc-2c0d-486d-8c8c-7ac3558cfd33 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:13:32.724392 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-60145ccc-2c0d-486d-8c8c-7ac3558cfd33 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/c7507aca-90a7-46ed-9bf0-2a162db3bfc7" status: 200 len: 1266 microversion: 2.1 time: 0.171695 Oct 30 01:13:32.725185 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 680/1382] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:13:32 2022] GET /compute/v2.1/servers/c7507aca-90a7-46ed-9bf0-2a162db3bfc7 => generated 1266 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:13:33.750646 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-bc4ce834-a2c2-483c-aa1f-661c05c32afa tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:13:33.751377 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-bc4ce834-a2c2-483c-aa1f-661c05c32afa tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: c7507aca-90a7-46ed-9bf0-2a162db3bfc7] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:13:33.760472 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bc4ce834-a2c2-483c-aa1f-661c05c32afa tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:33.761169 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bc4ce834-a2c2-483c-aa1f-661c05c32afa tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:33.761885 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bc4ce834-a2c2-483c-aa1f-661c05c32afa tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:33.763057 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bc4ce834-a2c2-483c-aa1f-661c05c32afa tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:33.763758 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bc4ce834-a2c2-483c-aa1f-661c05c32afa tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:33.764446 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-bc4ce834-a2c2-483c-aa1f-661c05c32afa tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:33.904288 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-bc4ce834-a2c2-483c-aa1f-661c05c32afa tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:13:33.912209 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-bc4ce834-a2c2-483c-aa1f-661c05c32afa tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/c7507aca-90a7-46ed-9bf0-2a162db3bfc7" status: 200 len: 1266 microversion: 2.1 time: 0.167467 Oct 30 01:13:33.912787 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 703/1383] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:13:33 2022] GET /compute/v2.1/servers/c7507aca-90a7-46ed-9bf0-2a162db3bfc7 => generated 1266 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:13:34.937862 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-01640ad2-2e85-4fdc-bad0-b8cd01ec80f0 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:13:34.938519 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-01640ad2-2e85-4fdc-bad0-b8cd01ec80f0 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: c7507aca-90a7-46ed-9bf0-2a162db3bfc7] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:13:34.945564 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-01640ad2-2e85-4fdc-bad0-b8cd01ec80f0 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:34.946042 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-01640ad2-2e85-4fdc-bad0-b8cd01ec80f0 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:34.946467 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-01640ad2-2e85-4fdc-bad0-b8cd01ec80f0 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:34.947154 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-01640ad2-2e85-4fdc-bad0-b8cd01ec80f0 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:34.947596 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-01640ad2-2e85-4fdc-bad0-b8cd01ec80f0 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:34.948045 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-01640ad2-2e85-4fdc-bad0-b8cd01ec80f0 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:35.084841 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-01640ad2-2e85-4fdc-bad0-b8cd01ec80f0 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:13:35.093129 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-01640ad2-2e85-4fdc-bad0-b8cd01ec80f0 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/c7507aca-90a7-46ed-9bf0-2a162db3bfc7" status: 200 len: 1266 microversion: 2.1 time: 0.160383 Oct 30 01:13:35.093676 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 681/1384] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:13:34 2022] GET /compute/v2.1/servers/c7507aca-90a7-46ed-9bf0-2a162db3bfc7 => generated 1266 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:13:36.116324 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-b333c52a-4b8a-46f9-9704-b017038648c6 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:13:36.116905 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-b333c52a-4b8a-46f9-9704-b017038648c6 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: c7507aca-90a7-46ed-9bf0-2a162db3bfc7] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:13:36.123760 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b333c52a-4b8a-46f9-9704-b017038648c6 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:36.124255 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b333c52a-4b8a-46f9-9704-b017038648c6 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:36.124685 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b333c52a-4b8a-46f9-9704-b017038648c6 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:36.125390 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b333c52a-4b8a-46f9-9704-b017038648c6 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:36.125839 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b333c52a-4b8a-46f9-9704-b017038648c6 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:36.126258 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-b333c52a-4b8a-46f9-9704-b017038648c6 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:36.128039 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [req-8357cf09-25f6-45ae-b6b6-1b8f1b931e8f req-a31eefaa-2d87-4057-a0ec-21b6987e5da8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c7507aca-90a7-46ed-9bf0-2a162db3bfc7", "name": "network-vif-plugged", "status": "completed", "tag": "a40ffbd8-6a20-454e-a5cf-5c6347862ffe"}]} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:13:36.140435 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-8357cf09-25f6-45ae-b6b6-1b8f1b931e8f req-a31eefaa-2d87-4057-a0ec-21b6987e5da8 service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:36.140912 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-8357cf09-25f6-45ae-b6b6-1b8f1b931e8f req-a31eefaa-2d87-4057-a0ec-21b6987e5da8 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:36.141439 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-8357cf09-25f6-45ae-b6b6-1b8f1b931e8f req-a31eefaa-2d87-4057-a0ec-21b6987e5da8 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:36.155978 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.compute.server_external_events [req-8357cf09-25f6-45ae-b6b6-1b8f1b931e8f req-a31eefaa-2d87-4057-a0ec-21b6987e5da8 service nova] Creating event network-vif-plugged:a40ffbd8-6a20-454e-a5cf-5c6347862ffe for instance c7507aca-90a7-46ed-9bf0-2a162db3bfc7 on hv-cinder-maste Oct 30 01:13:36.161732 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [req-8357cf09-25f6-45ae-b6b6-1b8f1b931e8f req-a31eefaa-2d87-4057-a0ec-21b6987e5da8 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037756 Oct 30 01:13:36.162343 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 682/1385] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:13:36 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:13:36.277220 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-b333c52a-4b8a-46f9-9704-b017038648c6 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:13:36.284930 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-b333c52a-4b8a-46f9-9704-b017038648c6 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/c7507aca-90a7-46ed-9bf0-2a162db3bfc7" status: 200 len: 1266 microversion: 2.1 time: 0.172568 Oct 30 01:13:36.285489 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 704/1386] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:13:36 2022] GET /compute/v2.1/servers/c7507aca-90a7-46ed-9bf0-2a162db3bfc7 => generated 1266 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:13:37.312383 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-3e0aeddd-fedb-4aea-83e0-19a157fbdd02 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:13:37.313101 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-3e0aeddd-fedb-4aea-83e0-19a157fbdd02 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: c7507aca-90a7-46ed-9bf0-2a162db3bfc7] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:13:37.319969 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3e0aeddd-fedb-4aea-83e0-19a157fbdd02 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:37.320443 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3e0aeddd-fedb-4aea-83e0-19a157fbdd02 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:37.320864 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3e0aeddd-fedb-4aea-83e0-19a157fbdd02 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:37.321549 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3e0aeddd-fedb-4aea-83e0-19a157fbdd02 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:37.321992 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3e0aeddd-fedb-4aea-83e0-19a157fbdd02 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:37.322391 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-3e0aeddd-fedb-4aea-83e0-19a157fbdd02 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:37.460950 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-3e0aeddd-fedb-4aea-83e0-19a157fbdd02 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:13:37.469421 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-3e0aeddd-fedb-4aea-83e0-19a157fbdd02 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/c7507aca-90a7-46ed-9bf0-2a162db3bfc7" status: 200 len: 1266 microversion: 2.1 time: 0.163836 Oct 30 01:13:37.469992 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 683/1387] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:13:37 2022] GET /compute/v2.1/servers/c7507aca-90a7-46ed-9bf0-2a162db3bfc7 => generated 1266 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:13:38.492373 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-286b6fe0-0427-4411-ad64-3a8d9e2defa2 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:13:38.492768 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-286b6fe0-0427-4411-ad64-3a8d9e2defa2 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: c7507aca-90a7-46ed-9bf0-2a162db3bfc7] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:13:38.499301 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-286b6fe0-0427-4411-ad64-3a8d9e2defa2 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:38.499816 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-286b6fe0-0427-4411-ad64-3a8d9e2defa2 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:38.500254 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-286b6fe0-0427-4411-ad64-3a8d9e2defa2 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:38.500949 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-286b6fe0-0427-4411-ad64-3a8d9e2defa2 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:38.501408 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-286b6fe0-0427-4411-ad64-3a8d9e2defa2 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:38.501830 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-286b6fe0-0427-4411-ad64-3a8d9e2defa2 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:38.638819 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-286b6fe0-0427-4411-ad64-3a8d9e2defa2 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:13:38.647264 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-286b6fe0-0427-4411-ad64-3a8d9e2defa2 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/c7507aca-90a7-46ed-9bf0-2a162db3bfc7" status: 200 len: 1453 microversion: 2.1 time: 0.159308 Oct 30 01:13:38.647911 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 705/1388] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:13:38 2022] GET /compute/v2.1/servers/c7507aca-90a7-46ed-9bf0-2a162db3bfc7 => generated 1453 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:13:38.669733 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-fab043dd-ef62-4296-92dd-bf72f5829e98 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:13:38.670346 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-fab043dd-ef62-4296-92dd-bf72f5829e98 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: c7507aca-90a7-46ed-9bf0-2a162db3bfc7] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:13:38.677088 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-fab043dd-ef62-4296-92dd-bf72f5829e98 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:38.677566 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-fab043dd-ef62-4296-92dd-bf72f5829e98 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:38.677997 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-fab043dd-ef62-4296-92dd-bf72f5829e98 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:38.678678 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-fab043dd-ef62-4296-92dd-bf72f5829e98 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:38.679123 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-fab043dd-ef62-4296-92dd-bf72f5829e98 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:38.679536 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-fab043dd-ef62-4296-92dd-bf72f5829e98 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:38.812205 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-fab043dd-ef62-4296-92dd-bf72f5829e98 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:13:38.820122 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-fab043dd-ef62-4296-92dd-bf72f5829e98 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/c7507aca-90a7-46ed-9bf0-2a162db3bfc7" status: 200 len: 1453 microversion: 2.1 time: 0.154458 Oct 30 01:13:38.820704 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 684/1389] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:13:38 2022] GET /compute/v2.1/servers/c7507aca-90a7-46ed-9bf0-2a162db3bfc7 => generated 1453 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:13:40.600785 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [req-2793539f-cb4a-4941-88a0-e1d0af2e22c2 req-faa29847-97c6-4a02-843e-041928a0b451 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c7507aca-90a7-46ed-9bf0-2a162db3bfc7", "tag": "a40ffbd8-6a20-454e-a5cf-5c6347862ffe"}]} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:13:40.610903 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-2793539f-cb4a-4941-88a0-e1d0af2e22c2 req-faa29847-97c6-4a02-843e-041928a0b451 service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:13:40.611381 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-2793539f-cb4a-4941-88a0-e1d0af2e22c2 req-faa29847-97c6-4a02-843e-041928a0b451 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:13:40.611836 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-2793539f-cb4a-4941-88a0-e1d0af2e22c2 req-faa29847-97c6-4a02-843e-041928a0b451 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:13:40.625392 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.compute.server_external_events [req-2793539f-cb4a-4941-88a0-e1d0af2e22c2 req-faa29847-97c6-4a02-843e-041928a0b451 service nova] Creating event network-changed:a40ffbd8-6a20-454e-a5cf-5c6347862ffe for instance c7507aca-90a7-46ed-9bf0-2a162db3bfc7 on hv-cinder-maste Oct 30 01:13:40.630517 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [req-2793539f-cb4a-4941-88a0-e1d0af2e22c2 req-faa29847-97c6-4a02-843e-041928a0b451 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033427 Oct 30 01:13:40.631353 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 706/1390] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:13:40 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:15:03.431542 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR oslo.messaging._drivers.impl_rabbit [-] [a5e1fd7f-f91d-4de5-b6b6-8792570242d8] AMQP server on 10.222.0.16:5672 is unreachable: . Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: Oct 30 01:15:03.434676 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_api:SELECT=36,nova_api:INSERT=3 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:15:03.435426 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell1:SELECT=55 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:15:03.436037 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-2e822dfa-1477-4f54-9b24-3d234fa24d9c tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:15:03.436837 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-2e822dfa-1477-4f54-9b24-3d234fa24d9c tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:15:03.439200 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_api:SELECT=27 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:15:03.439894 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell1:SELECT=67 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:15:03.441928 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [req-32d3c7f9-0ed2-4920-993a-d8e16f12a04e req-9f45337f-1f55-4c2f-89f3-2eb30d930fdc service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4444bade-70b8-4cef-9037-02cc5773a5d5", "tag": "4025698b-6028-4fe4-ae38-07266f391e05"}]} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:15:03.446850 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2e822dfa-1477-4f54-9b24-3d234fa24d9c tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:15:03.447755 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2e822dfa-1477-4f54-9b24-3d234fa24d9c tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:15:03.448274 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2e822dfa-1477-4f54-9b24-3d234fa24d9c tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:15:03.448976 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2e822dfa-1477-4f54-9b24-3d234fa24d9c tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:15:03.449489 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2e822dfa-1477-4f54-9b24-3d234fa24d9c tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:15:03.450226 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-2e822dfa-1477-4f54-9b24-3d234fa24d9c tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:15:03.453661 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-32d3c7f9-0ed2-4920-993a-d8e16f12a04e req-9f45337f-1f55-4c2f-89f3-2eb30d930fdc service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:15:03.454144 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-32d3c7f9-0ed2-4920-993a-d8e16f12a04e req-9f45337f-1f55-4c2f-89f3-2eb30d930fdc service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:15:03.454604 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-32d3c7f9-0ed2-4920-993a-d8e16f12a04e req-9f45337f-1f55-4c2f-89f3-2eb30d930fdc service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:15:03.468275 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.compute.server_external_events [req-32d3c7f9-0ed2-4920-993a-d8e16f12a04e req-9f45337f-1f55-4c2f-89f3-2eb30d930fdc service nova] Creating event network-changed:4025698b-6028-4fe4-ae38-07266f391e05 for instance 4444bade-70b8-4cef-9037-02cc5773a5d5 on hv-cinder-maste Oct 30 01:15:03.473562 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [req-32d3c7f9-0ed2-4920-993a-d8e16f12a04e req-9f45337f-1f55-4c2f-89f3-2eb30d930fdc service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.041092 Oct 30 01:15:03.474540 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 707/1391] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:15:03 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:15:03.479728 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-2e822dfa-1477-4f54-9b24-3d234fa24d9c tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] Going to try to terminate instance {{(pid=118962) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 30 01:15:03.568439 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-2e822dfa-1477-4f54-9b24-3d234fa24d9c tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "DELETE /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5" status: 204 len: 0 microversion: 2.1 time: 0.138312 Oct 30 01:15:03.569745 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 685/1392] 10.222.0.16 () {58 vars in 1239 bytes} [Sun Oct 30 01:15:03 2022] DELETE /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 30 01:15:03.579369 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-332e883f-a4fd-41fc-8901-53c0c2e86652 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:15:03.580266 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-332e883f-a4fd-41fc-8901-53c0c2e86652 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:15:03.586684 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-332e883f-a4fd-41fc-8901-53c0c2e86652 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:15:03.587238 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-332e883f-a4fd-41fc-8901-53c0c2e86652 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:15:03.587790 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-332e883f-a4fd-41fc-8901-53c0c2e86652 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:15:03.588557 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-332e883f-a4fd-41fc-8901-53c0c2e86652 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:15:03.589074 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-332e883f-a4fd-41fc-8901-53c0c2e86652 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:15:03.589557 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-332e883f-a4fd-41fc-8901-53c0c2e86652 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:15:03.726092 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-332e883f-a4fd-41fc-8901-53c0c2e86652 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baeaad0029f1455497fc3aea467dfc47', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f281bf8af484f7a88df7dc8ac06d887', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:15:03.734006 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-332e883f-a4fd-41fc-8901-53c0c2e86652 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5" status: 200 len: 1570 microversion: 2.1 time: 0.159370 Oct 30 01:15:03.734570 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 708/1393] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:15:03 2022] GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5 => generated 1570 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:15:04.757842 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-4cc7491f-5ed4-4279-906f-597b966ec38a tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:15:04.758515 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-4cc7491f-5ed4-4279-906f-597b966ec38a tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:15:04.766129 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4cc7491f-5ed4-4279-906f-597b966ec38a tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:15:04.766613 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4cc7491f-5ed4-4279-906f-597b966ec38a tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:15:04.767041 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4cc7491f-5ed4-4279-906f-597b966ec38a tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:15:04.767747 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4cc7491f-5ed4-4279-906f-597b966ec38a tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:15:04.768191 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4cc7491f-5ed4-4279-906f-597b966ec38a tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:15:04.768627 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-4cc7491f-5ed4-4279-906f-597b966ec38a tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:15:04.785292 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO oslo.messaging._drivers.impl_rabbit [-] [a5e1fd7f-f91d-4de5-b6b6-8792570242d8] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 35456. Oct 30 01:15:04.911590 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-4cc7491f-5ed4-4279-906f-597b966ec38a tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baeaad0029f1455497fc3aea467dfc47', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f281bf8af484f7a88df7dc8ac06d887', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:15:04.922918 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-4cc7491f-5ed4-4279-906f-597b966ec38a tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5" status: 200 len: 1570 microversion: 2.1 time: 0.169478 Oct 30 01:15:04.923490 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 686/1394] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:15:04 2022] GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5 => generated 1570 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:15:05.946677 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-943f1a9b-4472-4358-a1f3-03d235e67e71 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:15:05.947275 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-943f1a9b-4472-4358-a1f3-03d235e67e71 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:15:05.954924 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-943f1a9b-4472-4358-a1f3-03d235e67e71 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:15:05.955116 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-943f1a9b-4472-4358-a1f3-03d235e67e71 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:15:05.955479 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-943f1a9b-4472-4358-a1f3-03d235e67e71 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:15:05.956217 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-943f1a9b-4472-4358-a1f3-03d235e67e71 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:15:05.956672 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-943f1a9b-4472-4358-a1f3-03d235e67e71 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:15:05.957093 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-943f1a9b-4472-4358-a1f3-03d235e67e71 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:15:06.029042 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [req-de4a4552-f5f7-4ac2-ae93-04c0c957412d req-647d58df-9422-48ca-86a6-aed05dc1f10e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4444bade-70b8-4cef-9037-02cc5773a5d5", "name": "network-vif-deleted", "tag": "4025698b-6028-4fe4-ae38-07266f391e05"}]} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:15:06.029485 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-943f1a9b-4472-4358-a1f3-03d235e67e71 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baeaad0029f1455497fc3aea467dfc47', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f281bf8af484f7a88df7dc8ac06d887', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:15:06.038676 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-943f1a9b-4472-4358-a1f3-03d235e67e71 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5" status: 200 len: 1384 microversion: 2.1 time: 0.095997 Oct 30 01:15:06.039497 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 709/1395] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:15:05 2022] GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5 => generated 1384 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:15:06.041069 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-de4a4552-f5f7-4ac2-ae93-04c0c957412d req-647d58df-9422-48ca-86a6-aed05dc1f10e service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:15:06.041558 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-de4a4552-f5f7-4ac2-ae93-04c0c957412d req-647d58df-9422-48ca-86a6-aed05dc1f10e service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:15:06.041993 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-de4a4552-f5f7-4ac2-ae93-04c0c957412d req-647d58df-9422-48ca-86a6-aed05dc1f10e service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:15:06.054894 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.compute.server_external_events [req-de4a4552-f5f7-4ac2-ae93-04c0c957412d req-647d58df-9422-48ca-86a6-aed05dc1f10e service nova] Creating event network-vif-deleted:4025698b-6028-4fe4-ae38-07266f391e05 for instance 4444bade-70b8-4cef-9037-02cc5773a5d5 on hv-cinder-maste Oct 30 01:15:06.060205 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [req-de4a4552-f5f7-4ac2-ae93-04c0c957412d req-647d58df-9422-48ca-86a6-aed05dc1f10e service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034714 Oct 30 01:15:06.060933 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 687/1396] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:15:06 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 30 01:15:07.062821 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-e7c32886-e581-4886-968d-70aba35f9dc0 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:15:07.063463 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-e7c32886-e581-4886-968d-70aba35f9dc0 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:15:07.070399 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e7c32886-e581-4886-968d-70aba35f9dc0 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:15:07.070891 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e7c32886-e581-4886-968d-70aba35f9dc0 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:15:07.071325 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e7c32886-e581-4886-968d-70aba35f9dc0 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:15:07.072087 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e7c32886-e581-4886-968d-70aba35f9dc0 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:15:07.072539 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e7c32886-e581-4886-968d-70aba35f9dc0 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:15:07.072978 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e7c32886-e581-4886-968d-70aba35f9dc0 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:15:07.107320 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e7c32886-e581-4886-968d-70aba35f9dc0 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:15:07.107834 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e7c32886-e581-4886-968d-70aba35f9dc0 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:15:07.108266 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e7c32886-e581-4886-968d-70aba35f9dc0 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:15:07.108706 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.objects.instance [None req-e7c32886-e581-4886-968d-70aba35f9dc0 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lazy-loading 'fault' on Instance uuid 4444bade-70b8-4cef-9037-02cc5773a5d5 {{(pid=118961) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 30 01:15:07.151666 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-e7c32886-e581-4886-968d-70aba35f9dc0 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baeaad0029f1455497fc3aea467dfc47', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f281bf8af484f7a88df7dc8ac06d887', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:15:07.161051 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-e7c32886-e581-4886-968d-70aba35f9dc0 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5" status: 200 len: 1225 microversion: 2.1 time: 0.102176 Oct 30 01:15:07.161716 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 710/1397] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:15:07 2022] GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5 => generated 1225 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:15:08.189331 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-e955c10c-a35f-445b-a594-1a84f27abf89 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:15:08.190558 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-e955c10c-a35f-445b-a594-1a84f27abf89 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] [instance: 4444bade-70b8-4cef-9037-02cc5773a5d5] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:15:08.197539 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e955c10c-a35f-445b-a594-1a84f27abf89 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:15:08.198019 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e955c10c-a35f-445b-a594-1a84f27abf89 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:15:08.198420 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e955c10c-a35f-445b-a594-1a84f27abf89 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:15:08.199101 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e955c10c-a35f-445b-a594-1a84f27abf89 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:15:08.199539 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e955c10c-a35f-445b-a594-1a84f27abf89 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:15:08.199995 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-e955c10c-a35f-445b-a594-1a84f27abf89 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:15:08.214295 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.wsgi [None req-e955c10c-a35f-445b-a594-1a84f27abf89 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] HTTP exception thrown: Instance 4444bade-70b8-4cef-9037-02cc5773a5d5 could not be found. Oct 30 01:15:08.215970 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-e955c10c-a35f-445b-a594-1a84f27abf89 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Request method failure captured: Oct 30 01:15:08.215970 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: request: GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5 HTTP/1.1 Oct 30 01:15:08.215970 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept: application/json Oct 30 01:15:08.215970 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept-Encoding: identity Oct 30 01:15:08.215970 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Connection: close Oct 30 01:15:08.215970 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Length: 0 Oct 30 01:15:08.215970 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Type: application/json Oct 30 01:15:08.215970 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Host: 10.222.0.16 Oct 30 01:15:08.215970 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Openstack-System-Scope: None Oct 30 01:15:08.215970 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: User-Agent: python-urllib3/1.26.12 Oct 30 01:15:08.215970 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Auth-Token: gAAAAABjXc7QE_3uFUwPqDZFQ5BDzyl7E6-kmoS9coTOoxvgdyUeBRW9krz4WW4cMVZJw-UFwoXTelEd-8Oz8T4Nl5bPvcczJhRmWkb0MRhDwxSHnohkMc5PHpCoxGAeTDK9F4z-r75F0QbRVdvLo2jeW8LK8Q1vCe78lOQO7xSUWJRrk6qahaE Oct 30 01:15:08.215970 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Id: None Oct 30 01:15:08.215970 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Name: None Oct 30 01:15:08.215970 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Identity-Status: Confirmed Oct 30 01:15:08.215970 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Is-Admin-Project: True Oct 30 01:15:08.215970 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Id: default Oct 30 01:15:08.215970 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Name: Default Oct 30 01:15:08.215970 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Id: 4f281bf8af484f7a88df7dc8ac06d887 Oct 30 01:15:08.215970 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Name: tempest-TestShelveInstance-1408928340 Oct 30 01:15:08.215970 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Role: reader,member Oct 30 01:15:08.215970 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Roles: reader,member Oct 30 01:15:08.217763 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_4f281bf8af484f7a88df7dc8ac06d887"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/4f281bf8af484f7a88df7dc8ac06d887"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/4f281bf8af484f7a88df7dc8ac06d887"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/4f281bf8af484f7a88df7dc8ac06d887"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 01:15:08.217763 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant: tempest-TestShelveInstance-1408928340 Oct 30 01:15:08.217763 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Id: 4f281bf8af484f7a88df7dc8ac06d887 Oct 30 01:15:08.217763 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Name: tempest-TestShelveInstance-1408928340 Oct 30 01:15:08.217763 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User: tempest-TestShelveInstance-1408928340-project Oct 30 01:15:08.217763 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Id: default Oct 30 01:15:08.217763 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Name: Default Oct 30 01:15:08.217763 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Id: baeaad0029f1455497fc3aea467dfc47 Oct 30 01:15:08.217763 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Name: tempest-TestShelveInstance-1408928340-project Oct 30 01:15:08.217763 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: method: > Oct 30 01:15:08.217763 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: exception: Instance 4444bade-70b8-4cef-9037-02cc5773a5d5 could not be found. Oct 30 01:15:08.217763 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: action_args: {'id': '4444bade-70b8-4cef-9037-02cc5773a5d5'} Oct 30 01:15:08.218918 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 01:15:08.218918 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:15:08.218918 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 30 01:15:08.218918 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 30 01:15:08.218918 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 30 01:15:08.218918 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 30 01:15:08.218918 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 30 01:15:08.218918 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 30 01:15:08.218918 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 30 01:15:08.218918 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 30 01:15:08.218918 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4444bade-70b8-4cef-9037-02cc5773a5d5 could not be found. Oct 30 01:15:08.218918 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:15:08.218918 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:15:08.218918 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:15:08.218918 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:15:08.218918 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:15:08.218918 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:15:08.218918 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 01:15:08.218918 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 01:15:08.218918 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 01:15:08.220670 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 01:15:08.220670 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 30 01:15:08.220670 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 30 01:15:08.220670 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 30 01:15:08.220670 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 30 01:15:08.220670 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 30 01:15:08.220670 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 30 01:15:08.220670 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4444bade-70b8-4cef-9037-02cc5773a5d5 could not be found. Oct 30 01:15:08.220670 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:15:08.220670 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:15:08.220670 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:15:08.220670 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:15:08.220670 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:15:08.220670 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:15:08.220670 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 01:15:08.220670 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 01:15:08.220670 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4444bade-70b8-4cef-9037-02cc5773a5d5 could not be found. Oct 30 01:15:08.220670 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:15:08.222186 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-e955c10c-a35f-445b-a594-1a84f27abf89 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Returning 404 to user: Instance 4444bade-70b8-4cef-9037-02cc5773a5d5 could not be found. {{(pid=118962) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 01:15:08.222186 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-e955c10c-a35f-445b-a594-1a84f27abf89 tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5" status: 404 len: 111 microversion: 2.1 time: 0.035251 Oct 30 01:15:08.222186 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 688/1398] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:15:08 2022] GET /compute/v2.1/servers/4444bade-70b8-4cef-9037-02cc5773a5d5 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 30 01:15:08.382854 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-bab800db-5d09-4c0b-a71f-da2b71b2c4ef tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] Calling method '.version_select at 0x7f0eaba28dc0>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:15:08.408233 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-bab800db-5d09-4c0b-a71f-da2b71b2c4ef tempest-TestShelveInstance-1408928340 tempest-TestShelveInstance-1408928340-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1365136734" status: 202 len: 0 microversion: 2.1 time: 0.029047 Oct 30 01:15:08.408746 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 711/1399] 10.222.0.16 () {58 vars in 1254 bytes} [Sun Oct 30 01:15:08 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1365136734 => generated 0 bytes in 31 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 30 01:15:40.850637 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-794a02a1-c264-464f-a7ca-a24169f12809 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:15:40.851206 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_cell1:SELECT=22 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:15:40.851583 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-794a02a1-c264-464f-a7ca-a24169f12809 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: c7507aca-90a7-46ed-9bf0-2a162db3bfc7] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:15:40.852924 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG dbcounter [-] [118961] Writing DB stats nova_api:SELECT=13,nova_api:DELETE=1 {{(pid=118961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:15:40.853617 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_api:SELECT=13,nova_api:UPDATE=1 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:15:40.854389 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG dbcounter [-] [118962] Writing DB stats nova_cell1:SELECT=24,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118962) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:15:40.859657 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-794a02a1-c264-464f-a7ca-a24169f12809 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:15:40.860458 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-794a02a1-c264-464f-a7ca-a24169f12809 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:15:40.860901 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-794a02a1-c264-464f-a7ca-a24169f12809 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:15:40.861738 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-794a02a1-c264-464f-a7ca-a24169f12809 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:15:40.862128 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-794a02a1-c264-464f-a7ca-a24169f12809 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:15:40.862956 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-794a02a1-c264-464f-a7ca-a24169f12809 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:15:40.892077 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-794a02a1-c264-464f-a7ca-a24169f12809 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: c7507aca-90a7-46ed-9bf0-2a162db3bfc7] Going to try to terminate instance {{(pid=118962) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 30 01:15:40.990465 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-794a02a1-c264-464f-a7ca-a24169f12809 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "DELETE /compute/v2.1/servers/c7507aca-90a7-46ed-9bf0-2a162db3bfc7" status: 204 len: 0 microversion: 2.1 time: 0.143678 Oct 30 01:15:40.991740 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [req-4a40791c-8a0c-40d5-a88f-305350c3ec21 req-c9545fb2-1658-4c3b-b7b3-f5b1e14a4d74 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c7507aca-90a7-46ed-9bf0-2a162db3bfc7", "tag": "a40ffbd8-6a20-454e-a5cf-5c6347862ffe"}]} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:15:40.995361 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 689/1400] 10.222.0.16 () {58 vars in 1239 bytes} [Sun Oct 30 01:15:40 2022] DELETE /compute/v2.1/servers/c7507aca-90a7-46ed-9bf0-2a162db3bfc7 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 30 01:15:41.000562 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-301e4d7c-1a10-45d5-a0f6-a37155beaa8e tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:15:41.001892 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-301e4d7c-1a10-45d5-a0f6-a37155beaa8e tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: c7507aca-90a7-46ed-9bf0-2a162db3bfc7] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:15:41.002732 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-4a40791c-8a0c-40d5-a88f-305350c3ec21 req-c9545fb2-1658-4c3b-b7b3-f5b1e14a4d74 service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:15:41.003216 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-4a40791c-8a0c-40d5-a88f-305350c3ec21 req-c9545fb2-1658-4c3b-b7b3-f5b1e14a4d74 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:15:41.003646 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-4a40791c-8a0c-40d5-a88f-305350c3ec21 req-c9545fb2-1658-4c3b-b7b3-f5b1e14a4d74 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:15:41.007966 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-301e4d7c-1a10-45d5-a0f6-a37155beaa8e tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:15:41.008128 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-301e4d7c-1a10-45d5-a0f6-a37155beaa8e tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:15:41.008587 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-301e4d7c-1a10-45d5-a0f6-a37155beaa8e tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:15:41.009284 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-301e4d7c-1a10-45d5-a0f6-a37155beaa8e tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:15:41.009721 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-301e4d7c-1a10-45d5-a0f6-a37155beaa8e tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:15:41.010131 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-301e4d7c-1a10-45d5-a0f6-a37155beaa8e tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:15:41.016942 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.compute.server_external_events [req-4a40791c-8a0c-40d5-a88f-305350c3ec21 req-c9545fb2-1658-4c3b-b7b3-f5b1e14a4d74 service nova] Creating event network-changed:a40ffbd8-6a20-454e-a5cf-5c6347862ffe for instance c7507aca-90a7-46ed-9bf0-2a162db3bfc7 on hv-cinder-maste Oct 30 01:15:41.022839 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [req-4a40791c-8a0c-40d5-a88f-305350c3ec21 req-c9545fb2-1658-4c3b-b7b3-f5b1e14a4d74 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.173420 Oct 30 01:15:41.023523 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 712/1401] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:15:40 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:15:41.154331 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-301e4d7c-1a10-45d5-a0f6-a37155beaa8e tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:15:41.162311 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-301e4d7c-1a10-45d5-a0f6-a37155beaa8e tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/c7507aca-90a7-46ed-9bf0-2a162db3bfc7" status: 200 len: 1578 microversion: 2.1 time: 0.166133 Oct 30 01:15:41.162883 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 690/1402] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:15:40 2022] GET /compute/v2.1/servers/c7507aca-90a7-46ed-9bf0-2a162db3bfc7 => generated 1578 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:15:42.187254 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-8dfd66b6-addf-45f5-b23a-6c3bfa26b416 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:15:42.187895 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-8dfd66b6-addf-45f5-b23a-6c3bfa26b416 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: c7507aca-90a7-46ed-9bf0-2a162db3bfc7] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:15:42.194474 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8dfd66b6-addf-45f5-b23a-6c3bfa26b416 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:15:42.194961 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8dfd66b6-addf-45f5-b23a-6c3bfa26b416 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:15:42.195394 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8dfd66b6-addf-45f5-b23a-6c3bfa26b416 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:15:42.196133 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8dfd66b6-addf-45f5-b23a-6c3bfa26b416 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:15:42.196582 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8dfd66b6-addf-45f5-b23a-6c3bfa26b416 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:15:42.197005 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-8dfd66b6-addf-45f5-b23a-6c3bfa26b416 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:15:42.333402 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-8dfd66b6-addf-45f5-b23a-6c3bfa26b416 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:15:42.341582 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-8dfd66b6-addf-45f5-b23a-6c3bfa26b416 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/c7507aca-90a7-46ed-9bf0-2a162db3bfc7" status: 200 len: 1578 microversion: 2.1 time: 0.159472 Oct 30 01:15:42.342229 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 713/1403] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:15:42 2022] GET /compute/v2.1/servers/c7507aca-90a7-46ed-9bf0-2a162db3bfc7 => generated 1578 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:15:43.365181 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-7232b36c-c2b3-4fcf-90ca-47ed1059314c tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:15:43.365778 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-7232b36c-c2b3-4fcf-90ca-47ed1059314c tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: c7507aca-90a7-46ed-9bf0-2a162db3bfc7] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:15:43.372432 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7232b36c-c2b3-4fcf-90ca-47ed1059314c tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:15:43.372921 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7232b36c-c2b3-4fcf-90ca-47ed1059314c tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:15:43.373358 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7232b36c-c2b3-4fcf-90ca-47ed1059314c tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:15:43.374176 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7232b36c-c2b3-4fcf-90ca-47ed1059314c tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:15:43.374713 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7232b36c-c2b3-4fcf-90ca-47ed1059314c tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:15:43.375201 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-7232b36c-c2b3-4fcf-90ca-47ed1059314c tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:15:43.509016 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-7232b36c-c2b3-4fcf-90ca-47ed1059314c tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:15:43.517911 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-7232b36c-c2b3-4fcf-90ca-47ed1059314c tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/c7507aca-90a7-46ed-9bf0-2a162db3bfc7" status: 200 len: 1459 microversion: 2.1 time: 0.156712 Oct 30 01:15:43.523554 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 691/1404] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:15:43 2022] GET /compute/v2.1/servers/c7507aca-90a7-46ed-9bf0-2a162db3bfc7 => generated 1459 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:15:43.526550 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [req-fc0c64d3-2895-47ba-b348-42a260063b9d req-eed00ec1-412e-44c1-977f-b52837059f84 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c7507aca-90a7-46ed-9bf0-2a162db3bfc7", "name": "network-vif-deleted", "tag": "a40ffbd8-6a20-454e-a5cf-5c6347862ffe"}]} {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:15:43.538138 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-fc0c64d3-2895-47ba-b348-42a260063b9d req-eed00ec1-412e-44c1-977f-b52837059f84 service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:15:43.538610 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-fc0c64d3-2895-47ba-b348-42a260063b9d req-eed00ec1-412e-44c1-977f-b52837059f84 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:15:43.539032 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [req-fc0c64d3-2895-47ba-b348-42a260063b9d req-eed00ec1-412e-44c1-977f-b52837059f84 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:15:43.550647 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.compute.server_external_events [req-fc0c64d3-2895-47ba-b348-42a260063b9d req-eed00ec1-412e-44c1-977f-b52837059f84 service nova] Creating event network-vif-deleted:a40ffbd8-6a20-454e-a5cf-5c6347862ffe for instance c7507aca-90a7-46ed-9bf0-2a162db3bfc7 on hv-cinder-maste Oct 30 01:15:43.555998 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [req-fc0c64d3-2895-47ba-b348-42a260063b9d req-eed00ec1-412e-44c1-977f-b52837059f84 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034508 Oct 30 01:15:43.557445 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 714/1405] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:15:43 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:15:44.542059 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-d711c7e7-6063-48b2-acc6-028b591210ab tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:15:44.542740 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-d711c7e7-6063-48b2-acc6-028b591210ab tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: c7507aca-90a7-46ed-9bf0-2a162db3bfc7] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:15:44.549594 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d711c7e7-6063-48b2-acc6-028b591210ab tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:15:44.550111 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d711c7e7-6063-48b2-acc6-028b591210ab tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:15:44.550542 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d711c7e7-6063-48b2-acc6-028b591210ab tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:15:44.551231 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d711c7e7-6063-48b2-acc6-028b591210ab tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:15:44.551681 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d711c7e7-6063-48b2-acc6-028b591210ab tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:15:44.552131 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-d711c7e7-6063-48b2-acc6-028b591210ab tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:15:44.566528 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.wsgi [None req-d711c7e7-6063-48b2-acc6-028b591210ab tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] HTTP exception thrown: Instance c7507aca-90a7-46ed-9bf0-2a162db3bfc7 could not be found. Oct 30 01:15:44.568050 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-d711c7e7-6063-48b2-acc6-028b591210ab tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Request method failure captured: Oct 30 01:15:44.568050 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: request: GET /compute/v2.1/servers/c7507aca-90a7-46ed-9bf0-2a162db3bfc7 HTTP/1.1 Oct 30 01:15:44.568050 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept: application/json Oct 30 01:15:44.568050 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept-Encoding: identity Oct 30 01:15:44.568050 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Connection: close Oct 30 01:15:44.568050 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Length: 0 Oct 30 01:15:44.568050 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Type: application/json Oct 30 01:15:44.568050 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Host: 10.222.0.16 Oct 30 01:15:44.568050 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Openstack-System-Scope: None Oct 30 01:15:44.568050 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: User-Agent: python-urllib3/1.26.12 Oct 30 01:15:44.568050 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Auth-Token: gAAAAABjXc5Cle3iH9ItI2HXvwQRx6PZePcxrmkQWKXntioRsEcG8bGDbeJjw9hikhUg8ep4tjFQcKripNAxSYrAA9oY711XA708rHY3-1IRhoTotW6hXRFu4pB5gkihg7xD1caEs6KUEMpV4RpQtad_eiHzmSTNbKnNiaGaXF6jwXfx9XC17J0 Oct 30 01:15:44.568050 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Id: None Oct 30 01:15:44.568050 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Name: None Oct 30 01:15:44.568050 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Identity-Status: Confirmed Oct 30 01:15:44.568050 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Is-Admin-Project: True Oct 30 01:15:44.568050 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Id: default Oct 30 01:15:44.568050 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Name: Default Oct 30 01:15:44.568050 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Id: 238d71be4e094cb6b84f9a48a8a2c79c Oct 30 01:15:44.568050 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Name: tempest-TestVolumeBootPattern-653595049 Oct 30 01:15:44.568050 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Role: reader,member Oct 30 01:15:44.568050 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Roles: reader,member Oct 30 01:15:44.569816 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_238d71be4e094cb6b84f9a48a8a2c79c"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/238d71be4e094cb6b84f9a48a8a2c79c"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/238d71be4e094cb6b84f9a48a8a2c79c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/238d71be4e094cb6b84f9a48a8a2c79c"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 01:15:44.569816 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant: tempest-TestVolumeBootPattern-653595049 Oct 30 01:15:44.569816 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Id: 238d71be4e094cb6b84f9a48a8a2c79c Oct 30 01:15:44.569816 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Name: tempest-TestVolumeBootPattern-653595049 Oct 30 01:15:44.569816 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User: tempest-TestVolumeBootPattern-653595049-project Oct 30 01:15:44.569816 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Id: default Oct 30 01:15:44.569816 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Name: Default Oct 30 01:15:44.569816 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Id: 96e0cd00e4ef4a2da960aa5d4f66972a Oct 30 01:15:44.569816 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Name: tempest-TestVolumeBootPattern-653595049-project Oct 30 01:15:44.569816 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: method: > Oct 30 01:15:44.569816 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: exception: Instance c7507aca-90a7-46ed-9bf0-2a162db3bfc7 could not be found. Oct 30 01:15:44.569816 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: action_args: {'id': 'c7507aca-90a7-46ed-9bf0-2a162db3bfc7'} Oct 30 01:15:44.571109 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 01:15:44.571109 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:15:44.571109 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 30 01:15:44.571109 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 30 01:15:44.571109 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 30 01:15:44.571109 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 30 01:15:44.571109 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 30 01:15:44.571109 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 30 01:15:44.571109 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 30 01:15:44.571109 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 30 01:15:44.571109 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c7507aca-90a7-46ed-9bf0-2a162db3bfc7 could not be found. Oct 30 01:15:44.571109 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:15:44.571109 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:15:44.571109 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:15:44.571109 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:15:44.571109 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:15:44.571109 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:15:44.571109 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 01:15:44.571109 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 01:15:44.571109 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 01:15:44.572889 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 01:15:44.572889 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 30 01:15:44.572889 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 30 01:15:44.572889 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 30 01:15:44.572889 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 30 01:15:44.572889 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 30 01:15:44.572889 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 30 01:15:44.572889 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c7507aca-90a7-46ed-9bf0-2a162db3bfc7 could not be found. Oct 30 01:15:44.572889 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:15:44.572889 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:15:44.572889 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:15:44.572889 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:15:44.572889 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:15:44.572889 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:15:44.572889 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 01:15:44.572889 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 01:15:44.572889 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c7507aca-90a7-46ed-9bf0-2a162db3bfc7 could not be found. Oct 30 01:15:44.572889 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:15:44.575233 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-d711c7e7-6063-48b2-acc6-028b591210ab tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Returning 404 to user: Instance c7507aca-90a7-46ed-9bf0-2a162db3bfc7 could not be found. {{(pid=118962) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 01:15:44.575233 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-d711c7e7-6063-48b2-acc6-028b591210ab tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/c7507aca-90a7-46ed-9bf0-2a162db3bfc7" status: 404 len: 111 microversion: 2.1 time: 0.031336 Oct 30 01:15:44.575233 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 692/1406] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:15:44 2022] GET /compute/v2.1/servers/c7507aca-90a7-46ed-9bf0-2a162db3bfc7 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 30 01:15:47.686600 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-6fcfce82-4f6a-4fd7-b1da-bbbb782e38e9 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:15:47.687178 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-6fcfce82-4f6a-4fd7-b1da-bbbb782e38e9 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: 80d461e9-eac2-4d78-9e78-d37723f1543b] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:15:47.687510 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [req-14e8b45f-8447-45b7-9636-b91d024842f7 req-5bf79dd2-c4ea-4edf-bb88-014eba68b42a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "80d461e9-eac2-4d78-9e78-d37723f1543b", "tag": "49a23065-cebc-4735-bcae-f52ca512b794"}]} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:15:47.694991 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6fcfce82-4f6a-4fd7-b1da-bbbb782e38e9 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:15:47.695567 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6fcfce82-4f6a-4fd7-b1da-bbbb782e38e9 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:15:47.696107 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6fcfce82-4f6a-4fd7-b1da-bbbb782e38e9 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:15:47.696881 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6fcfce82-4f6a-4fd7-b1da-bbbb782e38e9 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:15:47.697295 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-14e8b45f-8447-45b7-9636-b91d024842f7 req-5bf79dd2-c4ea-4edf-bb88-014eba68b42a service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:15:47.697460 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6fcfce82-4f6a-4fd7-b1da-bbbb782e38e9 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:15:47.697750 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-14e8b45f-8447-45b7-9636-b91d024842f7 req-5bf79dd2-c4ea-4edf-bb88-014eba68b42a service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:15:47.698087 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-6fcfce82-4f6a-4fd7-b1da-bbbb782e38e9 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:15:47.698226 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-14e8b45f-8447-45b7-9636-b91d024842f7 req-5bf79dd2-c4ea-4edf-bb88-014eba68b42a service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:15:47.710683 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.compute.server_external_events [req-14e8b45f-8447-45b7-9636-b91d024842f7 req-5bf79dd2-c4ea-4edf-bb88-014eba68b42a service nova] Creating event network-changed:49a23065-cebc-4735-bcae-f52ca512b794 for instance 80d461e9-eac2-4d78-9e78-d37723f1543b on hv-cinder-maste Oct 30 01:15:47.716200 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [req-14e8b45f-8447-45b7-9636-b91d024842f7 req-5bf79dd2-c4ea-4edf-bb88-014eba68b42a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032033 Oct 30 01:15:47.717023 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 693/1407] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:15:47 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 30 01:15:47.726965 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-6fcfce82-4f6a-4fd7-b1da-bbbb782e38e9 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: 80d461e9-eac2-4d78-9e78-d37723f1543b] Going to try to terminate instance {{(pid=118961) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 30 01:15:47.823441 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-6fcfce82-4f6a-4fd7-b1da-bbbb782e38e9 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "DELETE /compute/v2.1/servers/80d461e9-eac2-4d78-9e78-d37723f1543b" status: 204 len: 0 microversion: 2.1 time: 0.141011 Oct 30 01:15:47.826009 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 715/1408] 10.222.0.16 () {58 vars in 1239 bytes} [Sun Oct 30 01:15:47 2022] DELETE /compute/v2.1/servers/80d461e9-eac2-4d78-9e78-d37723f1543b => generated 0 bytes in 145 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 30 01:15:47.835277 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-1a33ce5d-bd2d-4366-a74d-d62badf91767 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:15:47.836333 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-1a33ce5d-bd2d-4366-a74d-d62badf91767 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: 80d461e9-eac2-4d78-9e78-d37723f1543b] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:15:47.842779 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1a33ce5d-bd2d-4366-a74d-d62badf91767 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:15:47.843506 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1a33ce5d-bd2d-4366-a74d-d62badf91767 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:15:47.844150 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1a33ce5d-bd2d-4366-a74d-d62badf91767 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:15:47.845057 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1a33ce5d-bd2d-4366-a74d-d62badf91767 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:15:47.845708 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1a33ce5d-bd2d-4366-a74d-d62badf91767 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:15:47.846328 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-1a33ce5d-bd2d-4366-a74d-d62badf91767 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:15:47.989446 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.policy [None req-1a33ce5d-bd2d-4366-a74d-d62badf91767 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118962) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:15:47.997647 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-1a33ce5d-bd2d-4366-a74d-d62badf91767 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/80d461e9-eac2-4d78-9e78-d37723f1543b" status: 200 len: 1578 microversion: 2.1 time: 0.166578 Oct 30 01:15:47.998230 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 694/1409] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:15:47 2022] GET /compute/v2.1/servers/80d461e9-eac2-4d78-9e78-d37723f1543b => generated 1578 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:15:49.022307 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-e5157420-a7c1-42e1-8401-a639c8fc66de tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:15:49.023001 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-e5157420-a7c1-42e1-8401-a639c8fc66de tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: 80d461e9-eac2-4d78-9e78-d37723f1543b] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:15:49.029609 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e5157420-a7c1-42e1-8401-a639c8fc66de tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:15:49.030127 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e5157420-a7c1-42e1-8401-a639c8fc66de tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:15:49.030557 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e5157420-a7c1-42e1-8401-a639c8fc66de tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:15:49.031248 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e5157420-a7c1-42e1-8401-a639c8fc66de tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:15:49.031689 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e5157420-a7c1-42e1-8401-a639c8fc66de tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:15:49.032151 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-e5157420-a7c1-42e1-8401-a639c8fc66de tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:15:49.307904 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-e5157420-a7c1-42e1-8401-a639c8fc66de tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:15:49.316641 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-e5157420-a7c1-42e1-8401-a639c8fc66de tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/80d461e9-eac2-4d78-9e78-d37723f1543b" status: 200 len: 1578 microversion: 2.1 time: 0.298780 Oct 30 01:15:49.317225 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 716/1410] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:15:49 2022] GET /compute/v2.1/servers/80d461e9-eac2-4d78-9e78-d37723f1543b => generated 1578 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 30 01:15:50.220770 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [req-e5b091c9-b1cf-4b65-bf7b-ef504858fc36 req-3f1c2aa5-6f5d-4d5a-b158-7735acd96a27 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "80d461e9-eac2-4d78-9e78-d37723f1543b", "name": "network-vif-deleted", "tag": "49a23065-cebc-4735-bcae-f52ca512b794"}]} {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 30 01:15:50.231592 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-e5b091c9-b1cf-4b65-bf7b-ef504858fc36 req-3f1c2aa5-6f5d-4d5a-b158-7735acd96a27 service nova] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:15:50.232172 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-e5b091c9-b1cf-4b65-bf7b-ef504858fc36 req-3f1c2aa5-6f5d-4d5a-b158-7735acd96a27 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:15:50.232634 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [req-e5b091c9-b1cf-4b65-bf7b-ef504858fc36 req-3f1c2aa5-6f5d-4d5a-b158-7735acd96a27 service nova] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:15:50.243896 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.compute.server_external_events [req-e5b091c9-b1cf-4b65-bf7b-ef504858fc36 req-3f1c2aa5-6f5d-4d5a-b158-7735acd96a27 service nova] Creating event network-vif-deleted:49a23065-cebc-4735-bcae-f52ca512b794 for instance 80d461e9-eac2-4d78-9e78-d37723f1543b on hv-cinder-maste Oct 30 01:15:50.249066 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [req-e5b091c9-b1cf-4b65-bf7b-ef504858fc36 req-3f1c2aa5-6f5d-4d5a-b158-7735acd96a27 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032081 Oct 30 01:15:50.250072 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 695/1411] 10.222.0.16 () {64 vars in 1317 bytes} [Sun Oct 30 01:15:50 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 30 01:15:50.340265 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-61f8fd4d-8a7e-4edb-b66f-d9434855f2e3 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:15:50.341043 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-61f8fd4d-8a7e-4edb-b66f-d9434855f2e3 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: 80d461e9-eac2-4d78-9e78-d37723f1543b] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:15:50.349372 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-61f8fd4d-8a7e-4edb-b66f-d9434855f2e3 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:15:50.349991 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-61f8fd4d-8a7e-4edb-b66f-d9434855f2e3 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:15:50.350481 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-61f8fd4d-8a7e-4edb-b66f-d9434855f2e3 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:15:50.351187 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-61f8fd4d-8a7e-4edb-b66f-d9434855f2e3 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:15:50.351634 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-61f8fd4d-8a7e-4edb-b66f-d9434855f2e3 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:15:50.352334 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-61f8fd4d-8a7e-4edb-b66f-d9434855f2e3 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:15:50.415019 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.policy [None req-61f8fd4d-8a7e-4edb-b66f-d9434855f2e3 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e0cd00e4ef4a2da960aa5d4f66972a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '238d71be4e094cb6b84f9a48a8a2c79c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118961) authorize /opt/stack/nova/nova/policy.py:199}} Oct 30 01:15:50.423192 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-61f8fd4d-8a7e-4edb-b66f-d9434855f2e3 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/80d461e9-eac2-4d78-9e78-d37723f1543b" status: 200 len: 1225 microversion: 2.1 time: 0.087152 Oct 30 01:15:50.423816 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 717/1412] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:15:50 2022] GET /compute/v2.1/servers/80d461e9-eac2-4d78-9e78-d37723f1543b => generated 1225 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 30 01:15:51.450482 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-a8a6c1b6-f196-47ca-a99b-c5337b8b28cf tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:15:51.451359 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-a8a6c1b6-f196-47ca-a99b-c5337b8b28cf tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: 80d461e9-eac2-4d78-9e78-d37723f1543b] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:15:51.460367 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a8a6c1b6-f196-47ca-a99b-c5337b8b28cf tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:15:51.461003 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a8a6c1b6-f196-47ca-a99b-c5337b8b28cf tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:15:51.461638 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a8a6c1b6-f196-47ca-a99b-c5337b8b28cf tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:15:51.462548 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a8a6c1b6-f196-47ca-a99b-c5337b8b28cf tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:15:51.463144 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a8a6c1b6-f196-47ca-a99b-c5337b8b28cf tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:15:51.463855 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-a8a6c1b6-f196-47ca-a99b-c5337b8b28cf tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:15:51.478206 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.wsgi [None req-a8a6c1b6-f196-47ca-a99b-c5337b8b28cf tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] HTTP exception thrown: Instance 80d461e9-eac2-4d78-9e78-d37723f1543b could not be found. Oct 30 01:15:51.479902 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-a8a6c1b6-f196-47ca-a99b-c5337b8b28cf tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Request method failure captured: Oct 30 01:15:51.479902 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: request: GET /compute/v2.1/servers/80d461e9-eac2-4d78-9e78-d37723f1543b HTTP/1.1 Oct 30 01:15:51.479902 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept: application/json Oct 30 01:15:51.479902 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept-Encoding: identity Oct 30 01:15:51.479902 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Connection: close Oct 30 01:15:51.479902 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Length: 0 Oct 30 01:15:51.479902 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Type: application/json Oct 30 01:15:51.479902 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Host: 10.222.0.16 Oct 30 01:15:51.479902 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Openstack-System-Scope: None Oct 30 01:15:51.479902 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: User-Agent: python-urllib3/1.26.12 Oct 30 01:15:51.479902 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Auth-Token: gAAAAABjXc5Cle3iH9ItI2HXvwQRx6PZePcxrmkQWKXntioRsEcG8bGDbeJjw9hikhUg8ep4tjFQcKripNAxSYrAA9oY711XA708rHY3-1IRhoTotW6hXRFu4pB5gkihg7xD1caEs6KUEMpV4RpQtad_eiHzmSTNbKnNiaGaXF6jwXfx9XC17J0 Oct 30 01:15:51.479902 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Id: None Oct 30 01:15:51.479902 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Name: None Oct 30 01:15:51.479902 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Identity-Status: Confirmed Oct 30 01:15:51.479902 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Is-Admin-Project: True Oct 30 01:15:51.479902 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Id: default Oct 30 01:15:51.479902 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Name: Default Oct 30 01:15:51.479902 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Id: 238d71be4e094cb6b84f9a48a8a2c79c Oct 30 01:15:51.479902 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Name: tempest-TestVolumeBootPattern-653595049 Oct 30 01:15:51.479902 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Role: reader,member Oct 30 01:15:51.479902 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Roles: reader,member Oct 30 01:15:51.481609 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_238d71be4e094cb6b84f9a48a8a2c79c"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/238d71be4e094cb6b84f9a48a8a2c79c"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/238d71be4e094cb6b84f9a48a8a2c79c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/238d71be4e094cb6b84f9a48a8a2c79c"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 01:15:51.481609 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant: tempest-TestVolumeBootPattern-653595049 Oct 30 01:15:51.481609 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Id: 238d71be4e094cb6b84f9a48a8a2c79c Oct 30 01:15:51.481609 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Name: tempest-TestVolumeBootPattern-653595049 Oct 30 01:15:51.481609 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User: tempest-TestVolumeBootPattern-653595049-project Oct 30 01:15:51.481609 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Id: default Oct 30 01:15:51.481609 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Name: Default Oct 30 01:15:51.481609 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Id: 96e0cd00e4ef4a2da960aa5d4f66972a Oct 30 01:15:51.481609 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Name: tempest-TestVolumeBootPattern-653595049-project Oct 30 01:15:51.481609 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: method: > Oct 30 01:15:51.481609 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: exception: Instance 80d461e9-eac2-4d78-9e78-d37723f1543b could not be found. Oct 30 01:15:51.481609 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: action_args: {'id': '80d461e9-eac2-4d78-9e78-d37723f1543b'} Oct 30 01:15:51.482498 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 01:15:51.482498 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:15:51.482498 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 30 01:15:51.482498 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 30 01:15:51.482498 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 30 01:15:51.482498 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 30 01:15:51.482498 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 30 01:15:51.482498 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 30 01:15:51.482498 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 30 01:15:51.482498 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 30 01:15:51.482498 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 80d461e9-eac2-4d78-9e78-d37723f1543b could not be found. Oct 30 01:15:51.482498 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:15:51.482498 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:15:51.482498 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:15:51.482498 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:15:51.482498 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:15:51.482498 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:15:51.482498 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 01:15:51.482498 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 01:15:51.482498 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 01:15:51.483934 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 01:15:51.483934 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 30 01:15:51.483934 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 30 01:15:51.483934 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 30 01:15:51.483934 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 30 01:15:51.483934 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 30 01:15:51.483934 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 30 01:15:51.483934 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 80d461e9-eac2-4d78-9e78-d37723f1543b could not be found. Oct 30 01:15:51.483934 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:15:51.483934 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:15:51.483934 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:15:51.483934 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:15:51.483934 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:15:51.483934 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:15:51.483934 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 01:15:51.483934 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 01:15:51.483934 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 80d461e9-eac2-4d78-9e78-d37723f1543b could not be found. Oct 30 01:15:51.483934 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:15:51.485205 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-a8a6c1b6-f196-47ca-a99b-c5337b8b28cf tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Returning 404 to user: Instance 80d461e9-eac2-4d78-9e78-d37723f1543b could not be found. {{(pid=118962) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 01:15:51.485205 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-a8a6c1b6-f196-47ca-a99b-c5337b8b28cf tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/80d461e9-eac2-4d78-9e78-d37723f1543b" status: 404 len: 111 microversion: 2.1 time: 0.037421 Oct 30 01:15:51.485205 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 696/1413] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:15:51 2022] GET /compute/v2.1/servers/80d461e9-eac2-4d78-9e78-d37723f1543b => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 30 01:15:51.960234 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-23f72dda-433f-4e38-92d6-038e9980d114 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:15:51.960780 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.compute.api [None req-23f72dda-433f-4e38-92d6-038e9980d114 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: 91a8549d-4f75-4c98-9773-98efe4196889] Fetching instance by UUID {{(pid=118961) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:15:51.967419 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-23f72dda-433f-4e38-92d6-038e9980d114 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:15:51.967965 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-23f72dda-433f-4e38-92d6-038e9980d114 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:15:51.968450 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-23f72dda-433f-4e38-92d6-038e9980d114 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:15:51.969176 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-23f72dda-433f-4e38-92d6-038e9980d114 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:15:51.969675 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-23f72dda-433f-4e38-92d6-038e9980d114 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:15:51.970138 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG oslo_concurrency.lockutils [None req-23f72dda-433f-4e38-92d6-038e9980d114 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118961) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:15:51.983622 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.wsgi [None req-23f72dda-433f-4e38-92d6-038e9980d114 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] HTTP exception thrown: Instance 91a8549d-4f75-4c98-9773-98efe4196889 could not be found. Oct 30 01:15:51.985328 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-23f72dda-433f-4e38-92d6-038e9980d114 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Request method failure captured: Oct 30 01:15:51.985328 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: request: DELETE /compute/v2.1/servers/91a8549d-4f75-4c98-9773-98efe4196889 HTTP/1.1 Oct 30 01:15:51.985328 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Accept: application/json Oct 30 01:15:51.985328 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Accept-Encoding: identity Oct 30 01:15:51.985328 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Connection: close Oct 30 01:15:51.985328 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Content-Length: 0 Oct 30 01:15:51.985328 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Content-Type: application/json Oct 30 01:15:51.985328 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Host: 10.222.0.16 Oct 30 01:15:51.985328 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: Openstack-System-Scope: None Oct 30 01:15:51.985328 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: User-Agent: python-urllib3/1.26.12 Oct 30 01:15:51.985328 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Auth-Token: gAAAAABjXc5Cle3iH9ItI2HXvwQRx6PZePcxrmkQWKXntioRsEcG8bGDbeJjw9hikhUg8ep4tjFQcKripNAxSYrAA9oY711XA708rHY3-1IRhoTotW6hXRFu4pB5gkihg7xD1caEs6KUEMpV4RpQtad_eiHzmSTNbKnNiaGaXF6jwXfx9XC17J0 Oct 30 01:15:51.985328 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Domain-Id: None Oct 30 01:15:51.985328 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Domain-Name: None Oct 30 01:15:51.985328 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Identity-Status: Confirmed Oct 30 01:15:51.985328 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Is-Admin-Project: True Oct 30 01:15:51.985328 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Domain-Id: default Oct 30 01:15:51.985328 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Domain-Name: Default Oct 30 01:15:51.985328 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Id: 238d71be4e094cb6b84f9a48a8a2c79c Oct 30 01:15:51.985328 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Project-Name: tempest-TestVolumeBootPattern-653595049 Oct 30 01:15:51.985328 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Role: reader,member Oct 30 01:15:51.985328 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Roles: reader,member Oct 30 01:15:51.987117 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_238d71be4e094cb6b84f9a48a8a2c79c"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/238d71be4e094cb6b84f9a48a8a2c79c"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/238d71be4e094cb6b84f9a48a8a2c79c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/238d71be4e094cb6b84f9a48a8a2c79c"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 01:15:51.987117 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant: tempest-TestVolumeBootPattern-653595049 Oct 30 01:15:51.987117 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant-Id: 238d71be4e094cb6b84f9a48a8a2c79c Oct 30 01:15:51.987117 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-Tenant-Name: tempest-TestVolumeBootPattern-653595049 Oct 30 01:15:51.987117 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User: tempest-TestVolumeBootPattern-653595049-project Oct 30 01:15:51.987117 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Domain-Id: default Oct 30 01:15:51.987117 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Domain-Name: Default Oct 30 01:15:51.987117 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Id: 96e0cd00e4ef4a2da960aa5d4f66972a Oct 30 01:15:51.987117 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: X-User-Name: tempest-TestVolumeBootPattern-653595049-project Oct 30 01:15:51.987117 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: method: > Oct 30 01:15:51.987117 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: exception: Instance 91a8549d-4f75-4c98-9773-98efe4196889 could not be found. Oct 30 01:15:51.987117 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: action_args: {'id': '91a8549d-4f75-4c98-9773-98efe4196889'} Oct 30 01:15:51.988144 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 01:15:51.988144 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:15:51.988144 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 30 01:15:51.988144 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 30 01:15:51.988144 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 30 01:15:51.988144 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 30 01:15:51.988144 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 30 01:15:51.988144 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 30 01:15:51.988144 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 30 01:15:51.988144 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 30 01:15:51.988144 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 91a8549d-4f75-4c98-9773-98efe4196889 could not be found. Oct 30 01:15:51.988144 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:15:51.988144 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:15:51.988144 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:15:51.988144 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:15:51.988144 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:15:51.988144 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:15:51.988144 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 01:15:51.988144 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 01:15:51.988144 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 01:15:51.989833 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 01:15:51.989833 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 30 01:15:51.989833 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 30 01:15:51.989833 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 30 01:15:51.989833 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 30 01:15:51.989833 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 30 01:15:51.989833 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 30 01:15:51.989833 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 30 01:15:51.989833 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 30 01:15:51.989833 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 91a8549d-4f75-4c98-9773-98efe4196889 could not be found. Oct 30 01:15:51.989833 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:15:51.989833 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:15:51.989833 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:15:51.989833 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:15:51.989833 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:15:51.989833 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:15:51.989833 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 01:15:51.989833 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 01:15:51.989833 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 91a8549d-4f75-4c98-9773-98efe4196889 could not be found. Oct 30 01:15:51.989833 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: ERROR nova.api.openstack.wsgi Oct 30 01:15:51.991794 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-23f72dda-433f-4e38-92d6-038e9980d114 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Returning 404 to user: Instance 91a8549d-4f75-4c98-9773-98efe4196889 could not be found. {{(pid=118961) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 01:15:51.991794 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-23f72dda-433f-4e38-92d6-038e9980d114 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "DELETE /compute/v2.1/servers/91a8549d-4f75-4c98-9773-98efe4196889" status: 404 len: 111 microversion: 2.1 time: 0.030418 Oct 30 01:15:51.991794 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 718/1414] 10.222.0.16 () {58 vars in 1239 bytes} [Sun Oct 30 01:15:51 2022] DELETE /compute/v2.1/servers/91a8549d-4f75-4c98-9773-98efe4196889 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 30 01:15:51.995642 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-c80d63a3-4661-45fd-a6aa-d19043f05662 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '>' {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:15:51.996311 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.compute.api [None req-c80d63a3-4661-45fd-a6aa-d19043f05662 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] [instance: 91a8549d-4f75-4c98-9773-98efe4196889] Fetching instance by UUID {{(pid=118962) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 30 01:15:52.002680 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c80d63a3-4661-45fd-a6aa-d19043f05662 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:15:52.003159 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c80d63a3-4661-45fd-a6aa-d19043f05662 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:15:52.003595 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c80d63a3-4661-45fd-a6aa-d19043f05662 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:15:52.004316 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c80d63a3-4661-45fd-a6aa-d19043f05662 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Acquiring lock "a2b915ba-4703-4df7-87fc-723ef53ef868" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 30 01:15:52.004765 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c80d63a3-4661-45fd-a6aa-d19043f05662 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 30 01:15:52.005180 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG oslo_concurrency.lockutils [None req-c80d63a3-4661-45fd-a6aa-d19043f05662 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Lock "a2b915ba-4703-4df7-87fc-723ef53ef868" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118962) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 30 01:15:52.018686 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.wsgi [None req-c80d63a3-4661-45fd-a6aa-d19043f05662 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] HTTP exception thrown: Instance 91a8549d-4f75-4c98-9773-98efe4196889 could not be found. Oct 30 01:15:52.020234 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-c80d63a3-4661-45fd-a6aa-d19043f05662 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Request method failure captured: Oct 30 01:15:52.020234 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: request: GET /compute/v2.1/servers/91a8549d-4f75-4c98-9773-98efe4196889 HTTP/1.1 Oct 30 01:15:52.020234 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept: application/json Oct 30 01:15:52.020234 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Accept-Encoding: identity Oct 30 01:15:52.020234 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Connection: close Oct 30 01:15:52.020234 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Length: 0 Oct 30 01:15:52.020234 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Content-Type: application/json Oct 30 01:15:52.020234 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Host: 10.222.0.16 Oct 30 01:15:52.020234 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: Openstack-System-Scope: None Oct 30 01:15:52.020234 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: User-Agent: python-urllib3/1.26.12 Oct 30 01:15:52.020234 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Auth-Token: gAAAAABjXc5Cle3iH9ItI2HXvwQRx6PZePcxrmkQWKXntioRsEcG8bGDbeJjw9hikhUg8ep4tjFQcKripNAxSYrAA9oY711XA708rHY3-1IRhoTotW6hXRFu4pB5gkihg7xD1caEs6KUEMpV4RpQtad_eiHzmSTNbKnNiaGaXF6jwXfx9XC17J0 Oct 30 01:15:52.020234 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Id: None Oct 30 01:15:52.020234 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Domain-Name: None Oct 30 01:15:52.020234 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Identity-Status: Confirmed Oct 30 01:15:52.020234 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Is-Admin-Project: True Oct 30 01:15:52.020234 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Id: default Oct 30 01:15:52.020234 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Domain-Name: Default Oct 30 01:15:52.020234 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Id: 238d71be4e094cb6b84f9a48a8a2c79c Oct 30 01:15:52.020234 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Project-Name: tempest-TestVolumeBootPattern-653595049 Oct 30 01:15:52.020234 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Role: reader,member Oct 30 01:15:52.020234 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Roles: reader,member Oct 30 01:15:52.022150 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_238d71be4e094cb6b84f9a48a8a2c79c"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/238d71be4e094cb6b84f9a48a8a2c79c"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/238d71be4e094cb6b84f9a48a8a2c79c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/238d71be4e094cb6b84f9a48a8a2c79c"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Oct 30 01:15:52.022150 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant: tempest-TestVolumeBootPattern-653595049 Oct 30 01:15:52.022150 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Id: 238d71be4e094cb6b84f9a48a8a2c79c Oct 30 01:15:52.022150 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-Tenant-Name: tempest-TestVolumeBootPattern-653595049 Oct 30 01:15:52.022150 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User: tempest-TestVolumeBootPattern-653595049-project Oct 30 01:15:52.022150 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Id: default Oct 30 01:15:52.022150 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Domain-Name: Default Oct 30 01:15:52.022150 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Id: 96e0cd00e4ef4a2da960aa5d4f66972a Oct 30 01:15:52.022150 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: X-User-Name: tempest-TestVolumeBootPattern-653595049-project Oct 30 01:15:52.022150 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: method: > Oct 30 01:15:52.022150 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: exception: Instance 91a8549d-4f75-4c98-9773-98efe4196889 could not be found. Oct 30 01:15:52.022150 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: action_args: {'id': '91a8549d-4f75-4c98-9773-98efe4196889'} Oct 30 01:15:52.023151 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: {{(pid=118962) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 30 01:15:52.023151 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:15:52.023151 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 30 01:15:52.023151 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 30 01:15:52.023151 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 30 01:15:52.023151 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 30 01:15:52.023151 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 30 01:15:52.023151 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 30 01:15:52.023151 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 30 01:15:52.023151 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 30 01:15:52.023151 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 91a8549d-4f75-4c98-9773-98efe4196889 could not be found. Oct 30 01:15:52.023151 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:15:52.023151 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:15:52.023151 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:15:52.023151 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:15:52.023151 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:15:52.023151 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:15:52.023151 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 30 01:15:52.023151 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 30 01:15:52.023151 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 30 01:15:52.025078 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 30 01:15:52.025078 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 30 01:15:52.025078 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 30 01:15:52.025078 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 30 01:15:52.025078 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 30 01:15:52.025078 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 30 01:15:52.025078 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 30 01:15:52.025078 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 91a8549d-4f75-4c98-9773-98efe4196889 could not be found. Oct 30 01:15:52.025078 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:15:52.025078 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 30 01:15:52.025078 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:15:52.025078 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 30 01:15:52.025078 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 30 01:15:52.025078 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 30 01:15:52.025078 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 30 01:15:52.025078 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 30 01:15:52.025078 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 91a8549d-4f75-4c98-9773-98efe4196889 could not be found. Oct 30 01:15:52.025078 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: ERROR nova.api.openstack.wsgi Oct 30 01:15:52.026575 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: DEBUG nova.api.openstack.wsgi [None req-c80d63a3-4661-45fd-a6aa-d19043f05662 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Returning 404 to user: Instance 91a8549d-4f75-4c98-9773-98efe4196889 could not be found. {{(pid=118962) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 30 01:15:52.026575 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: INFO nova.api.openstack.requestlog [None req-c80d63a3-4661-45fd-a6aa-d19043f05662 tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "GET /compute/v2.1/servers/91a8549d-4f75-4c98-9773-98efe4196889" status: 404 len: 111 microversion: 2.1 time: 0.029977 Oct 30 01:15:52.026575 dv-cinder-master-3-iscsi devstack@n-api.service[118962]: [pid: 118962|app: 0|req: 697/1415] 10.222.0.16 () {58 vars in 1236 bytes} [Sun Oct 30 01:15:51 2022] GET /compute/v2.1/servers/91a8549d-4f75-4c98-9773-98efe4196889 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 30 01:15:53.348774 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: DEBUG nova.api.openstack.wsgi [None req-f6ce47a2-e0d4-4224-8085-f5027f41b35d tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] Calling method '.version_select at 0x7f0eb3424700>' {{(pid=118961) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 30 01:15:53.374986 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: INFO nova.api.openstack.requestlog [None req-f6ce47a2-e0d4-4224-8085-f5027f41b35d tempest-TestVolumeBootPattern-653595049 tempest-TestVolumeBootPattern-653595049-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1884997751" status: 202 len: 0 microversion: 2.1 time: 0.032095 Oct 30 01:15:53.375837 dv-cinder-master-3-iscsi devstack@n-api.service[118961]: [pid: 118961|app: 0|req: 719/1416] 10.222.0.16 () {58 vars in 1263 bytes} [Sun Oct 30 01:15:53 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1884997751 => generated 0 bytes in 34 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0)